JP5810824B2 - Program, method, and information processing apparatus - Google Patents

Description

  The present invention relates to processing according to an operation on a touch panel display device.

  2. Description of the Related Art Information processing devices or terminals that are operated by touching a touch panel placed on a display device with a finger or a touch pen and specifying an input position or coordinates on the display device are known. There is a multi-touch panel that can detect a plurality of contact positions at the same time.

  A certain known operation control device displays a first icon indicating the type of operation condition on the display unit, and when it is determined that the first icon is touched on the touch panel, the operation indicated by the first icon A second icon indicating specific options for the condition can be displayed at a position corresponding to the first operation. The operation control device further selects a specific option of the second icon corresponding to the second touch operation when determining that the second touch operation has been performed on the displayed second icon. . Thereby, an event of the selected option is generated.

  A method for handling touch events in known multi-touch devices displays one or more views on a display, each executing one or more software elements associated with a particular view. In that method, a multi-touch flag or an exclusive touch flag is associated with an individual view to receive one or more touches in one or more views. The method further associated one or more touch events describing the received touch with one or more views that were touched based on the values of the multi-touch flag and the exclusive touch flag. Selectively transmit to one or more software elements. Each view can be an exclusive view or a non-exclusive view, and touch events in the view can be ignored or recognized. Ignored touches do not need to be sent to the application. By selectively ignoring touches, simpler software elements can be realized.

JP 2010-204844 A JP 2009-2111704 A

  In a normal touch panel of an information processing apparatus, when a user activates a desired element (for example, a setting change function) of an application (for example, a web browser), the user first touches the software button for starting the application. Manipulate. Then, a list of software buttons associated with a plurality of elements in the application is displayed on the display device. The user touches and selects a software button indicating a desired element from the software button list. When such multiple software buttons are organized hierarchically, many touch operations are required to reach the lower-level software button associated with the desired application element. .

  The inventor can reduce the number of operations or the operation time by enabling the application element to be activated or the event to be generated to be determined based on the relative positional relationship between the two touch positions on the touch panel. I recognized.

  An object of the embodiment is to improve operability for activating an activation item in a touch panel display device.

According to one aspect of the embodiment, data representing the first contact position and the second contact position on the display screen of the touch panel display device is acquired from the touch panel display device, and the second contact position is software. If there-on button, the relative to that of the first contact position relative to the second contact position, and determining at least one of the relative positions of the bearing and distance, the software button which is stored in the storage device A process of referring to information representing an activation item associated with a position and determining to display or activate one or more activation items associated with the determined relative position; A program for causing an information processing apparatus to execute is provided.

  According to one aspect of the embodiment, the operability for activating the activation item in the touch panel display device can be improved.

FIG. 1A illustrates an example of a schematic configuration of the information processing apparatus according to the embodiment. FIG. 1B is a perspective view illustrating an example of a positional relationship between the display unit and the touch panel in the touch panel display device. FIG. 2 shows an example of a schematic configuration of the processor of the information processing apparatus. FIG. 3A shows an example of a table of activation items for each activation code. FIG. 3B is an activation code table for the combination of the icon button at the second touch position and the relative position of the first touch position with respect to the second touch position in the touch panel display device used with the table of FIG. 3A. An example is shown. 4A to 4D illustrate examples of operation procedures on the display screen of the touch panel display device in the simultaneous multiple touch operation mode according to the embodiment. 5A and 5B show an example of an operation procedure on a display screen of a touch panel display device in a time difference multiple touch operation mode using a timer according to the embodiment. 6A to 6D illustrate an example of an operation procedure on the display screen of the touch panel display device in another time difference multiple touch operation mode that does not use a timer according to the embodiment. An example is shown. FIG. 7A shows an example of a table of one or more activation items for each activation code. FIG. 7B is similar to FIG. 3B for the combination of the icon button at the second contact position and the relative position of the first contact position with respect to the second contact position in the touch panel display device used with the table of FIG. 7A. An example of a table of activation codes is shown. 8A to 8D show an example of an operation procedure on the display screen of the touch panel display device in the simultaneous multiple touch operation mode when selectable activation items are displayed according to the embodiment. 9A and 9B show an example of an operation procedure on a display screen of a touch panel display device in a time difference multiple touch operation mode using a timer when a selectable activation item is displayed according to the embodiment. 10A to 10D show an example of an operation procedure on the display screen of the touch panel display device in another time difference multiple touch operation mode that does not use a timer when selectable activation items are displayed according to the embodiment. Yes. FIG. 11A shows an example in which the positional relationship of the first contact position with respect to the second contact position is classified into four azimuth ranges. FIG. 11B shows an example in which the positional relationship of the first contact position with respect to the second contact position is classified into four distance ranges. FIG. 11C shows an example in which the positional relationship of the first contact position with respect to the second contact position is classified into eight azimuth ranges. FIG. 11D shows an example in which the positional relationship of the first contact position with respect to the second contact position is classified into a range of combinations of four directions and four distances. 12A to 12C illustrate an example of an operation procedure for activating a desired activation item in the information processing apparatus in the single touch operation mode. 13A and 13B show an example of an operation procedure for activating a desired activation item in the information processing apparatus in the multi-touch operation mode. FIG. 14A shows an example of a table of activation items for each activation code as applied to FIGS. 13A and 13B. FIG. 14B shows an activation code table associated with the combination of the icon button at the second contact position and the relative position of the first contact position with respect to the second contact position, as applied to FIGS. 13A and 13B. An example is shown. 15A to 15C show another example of an operation procedure for activating a desired activation item in the information processing apparatus in the single touch operation mode. 16A to 16E are modifications of the operation procedure of FIGS. 15A to 15C and show another example of the operation procedure for activating a desired activation item in the information processing apparatus in the single touch operation mode. . 17A and 17B show an example of an operation procedure for activating a desired activation item in the information processing apparatus in the multi-touch operation mode. FIGS. 18A to 18D are variations of the operation procedure of FIGS. 17A and 17B and show another example of the operation procedure for activating a desired activation item in the information processing apparatus in the multi-touch operation mode. FIG. 19A shows an example of a table of activation items for each activation code as applied to FIGS. 17A, 17B and 18A-18D. FIG. 19B shows the activation associated with the combination of the icon button at the second contact position and the relative position of the first contact position with respect to the second contact position, as applied to FIGS. 18A, 18B and 19A-19D. An example of a code table is shown. FIG. 20A shows another example of a table of activation items for each activation code. FIG. 20B shows another example of the activation code table associated with the combination of the icon button at the second contact position TP2 and the relative position of the first contact position with respect to the second contact position. FIG. 21A shows another example of a table of activation items for each activation code. FIG. 21B shows another example of the activation code table associated with the combination of the icon button at the second contact position TP2 and the relative position of the first contact position with respect to the second contact position. FIG. 22A shows another example of a table of activation items for each activation code. FIG. 22B shows another example of the table of activation codes associated with the combination of the icon button at the second contact position and the relative position of the first contact position with respect to the second contact position. FIG. 23A shows another example of a table of activation items for each activation code. FIG. 23B shows another example of the table of activation codes associated with the combination of the icon button at the second contact position TP2 and the relative position of the first contact position with respect to the second contact position. 24A to 24C show another example of an operation procedure for activating a desired activation item in the information processing apparatus in the single touch operation mode. 25A and 25B show another example of an operation procedure for activating a desired activation item in the information processing apparatus in the multi-touch operation mode. FIG. 26A shows an example of an activation item table for each activation code as applied to FIGS. 25A and 25B. FIG. 26B is a table of activation codes associated with the combination of the icon button at the second contact position TP2 and the relative position of the first contact position with respect to the second contact position, as applied to FIGS. 25A and 25B. An example is shown. FIG. 27 shows an example of a flowchart for determining the contact state of one or two contacts (touch) executed by the processor of the information processing apparatus or its contact detection unit in the simultaneous multiple touch operation mode. 28A to 28D show examples of flowcharts for determining each process according to the contact state of the first contact position, which is executed by the processor or the first contact state determination unit thereof. (Explained in Fig. 28A) (Explained in Fig. 28A) (Explained in Fig. 28A) FIGS. 29A to 29C show examples of flowcharts for determining each process according to the contact state of the second contact position, which is executed by the processor or the second contact state determination unit thereof. (Explained in Fig. 29A) (Explained in Fig. 29A) FIG. 30 is an example of a flowchart of processing executed by the processor or its relative position determination unit, activation item selection unit, activation item display unit, and auxiliary line display unit in response to the notification of FIGS. 28C, 29A and 29B. Is shown. FIG. 31A shows an example of a flowchart for processing executed by the processor or its relative position determination unit, activation item selection unit, and activation processing unit in response to FIG. 29C. FIG. 31B shows an example of a flowchart for processing executed by the processor or its activation processing unit corresponding to FIG. 29C. 32A to 32C show examples of screens for registering the correspondence between each icon / button and the relative position combination of the first contact position and the activation item. 33A and 33B are flowcharts for determining the contact state of one or two contacts (touch) executed by the processor of the information processing apparatus 10 or its contact detection unit in the time difference multiple touch operation mode using a timer. An example is shown. (Explained in Fig. 33A) FIG. 34 shows an example of a flowchart for determining processing according to the contact state of the first contact position, which is executed by the processor or its first contact state determination unit or the like. FIGS. 35A to 35C show examples of flowcharts for determining processing according to the contact state of the second contact position, which is executed by the processor or the second contact state determination unit thereof. (Explained in Fig. 35A) (Explained in Fig. 35A) FIG. 36 shows an example of a flowchart for switching the timer mode set by the user, which is executed by the processor or its timer mode setting unit in the time difference multiple touch operation mode without using the timer. FIG. 37 is an example of a flowchart for determining the contact state of one or two contacts (touch) executed by the processor of the information processing apparatus or its contact detection unit in the time difference multiple touch operation mode without using a timer. Show. 38A and 38B show still another example of the operation procedure on the display screen of the touch panel display device in the multiple touch operation mode according to the embodiment.

The objects and advantages of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the appended claims.
It is understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not intended to limit the invention.

  Non-limiting embodiments of the present invention will be described with reference to the drawings. In the drawings, similar components and elements have the same reference numerals.

  The inventor determines the number of operations or the operation time by determining an application element to be activated or an event to be generated based on a relative position between the first contact position on the touch panel and the second contact position on the software button. Recognized that it can be reduced.

  FIG. 1A illustrates an example of a schematic configuration of the information processing apparatus 10 according to the embodiment.

  1A, the information processing apparatus 10 includes a processor 102, a memory 104, an input / output interface (I / O) 108, an internal bus, and the like. The information processing apparatus 10 also includes a touch panel display device 110 and a storage device 130 such as a hard disk drive (HDD) or a flash memory coupled to the input / output interface 108. The information processing apparatus 10 may be further connectable to an external drive 128 for reading a recording medium.

  The information processing apparatus 10 may be a computer such as a personal computer or a device such as a portable terminal. Such a portable terminal may be, for example, a PDA (Personal Digital Assistant), a tablet terminal, a smartphone, or another portable phone. The information processing apparatus 10 includes, for example, a camera, a car navigation system, a ticket machine, a vending machine, a cash register, an automatic teller machine (ATM), a game machine, an audio device, a copying machine, a facsimile machine, a printer, Any device including a touch panel display device may be used.

  The processor 102 may be a CPU (Central Processing Unit) for a computer. The processor 102 and memory 104 are coupled to an input / output interface 108. The memory 104 includes, for example, a main storage device and a semiconductor memory.

  FIG. 1B is a schematic perspective view illustrating an example of a positional relationship between a display unit 112 such as a liquid crystal panel and a touch panel 114 in the touch panel display device 110.

  The touch panel display device 110 may include, for example, a display unit 112 such as a liquid crystal display unit, a touch panel 114, and a contact position-coordinate converter or circuit (not shown). A touch panel 114 is aligned and disposed on the display unit 112. The touch panel 114 generates the coordinates (x, y) of one or more positions touched by one or more fingers 116 and / or touch means such as the touch pen 118 and data representing the state and supplies the data to the processor 102. The contact position-coordinate converter may convert a physical quantity detection value representing one or more contact positions of the finger 116 or the touch pen 118 on the touch panel 114 into coordinate data and output the coordinate data.

  The information processing apparatus 10 may further include another input device (not shown) such as a keyboard, for example, a pointing device such as a mouse or a touch pad.

  The drive 128 is used to read a recording medium 129 such as an optical disc on which software is recorded. The software may include, for example, an OS (Operating System), application programs, and the like.

  The processor 102 may operate according to the above-described software stored in the memory 104 and / or the storage device 130 or a part thereof. The software described above may be read from the recording medium 129 by the drive 128 and installed in the information processing apparatus 10. Further, as an alternative, the processor 102 may be a dedicated processor that is implemented as an integrated circuit, for example, including at least a part of the software functions described above.

  The information processing apparatus 10 may further include a wireless communication unit 142 and a network interface (NW I / F) 144 coupled to an input / output interface (I / O) 108. The information processing apparatus 10 may be connected to a server apparatus (not shown) via the network interface 144 and further via the external network 5. The wireless communication unit 142 includes, for example, a short-range wireless communication device such as Bluetooth (registered trademark) standard, NFC (Near Field Communication), or wireless LAN (Local Area Network), and communicates with other devices. May be.

  The network 5 includes, for example, a LAN (Local Area Network), the Internet, a public switched telephone network (PSTN), a packet switched network (PSN), an ISDN (Integrated Services Digital Network), a mobile communication network, and / or a dedicated line. You may go out.

  FIG. 2 shows an example of a schematic configuration of the processor 102 of the information processing apparatus 10. The processor 102 includes a control unit 1020 and an application unit (or software execution unit) 1022. The processor 102 further includes a contact (touch) detection unit 1122, a first contact state determination unit 1124, a second contact state determination unit 1126, a timer 1128, and a timer mode setting unit 1130, or a part thereof. Also good. The processor 102 also includes an auxiliary line display unit 1132, a relative position determination unit 1134, an activation item selection unit or determination unit 1142, an activation item display unit 1144, an activation processing unit 1146, a correspondence setting unit 1160, and other processing. The portion 1060 or a part thereof may be included. The controller 1020 may supply control signals to the elements 1022, 1122-1146, and 1060 to control the operation of these elements. Elements 1022, 1122-1146 and 1060, or portions thereof, may be implemented as an application or software executed by processor 102.

  FIG. 3A shows an example of a table of activation items for each activation code.

  In FIG. 3A, an activation item is an item to be activated and represents an item, element or object to be activated in an application or software. The activation code is operation information, operation parameters or control information for activating, activating, activating, generating or calling an activation item. The activation item may be, for example, identification information representing a function, an operation mode, or a program of an application or software. The activation item may be condition information (for example, communication means, input information, search keyword, address) used for execution of an application or an element of software. Further, the activation item may be identification information representing a desired event (for example, a certain display screen, a certain information display) caused by execution of a certain element of an application or software.

  For example, the activation code MnKy1-1 is associated with the use communication medium “infrared” of the data transfer application as the activation item, and the activation code MnKy1-2 (activation item 2) is associated with any element. It is not done. For example, the activation code MnKy1-3 is associated with the use communication means “Bluetooth (registered trademark)” (Bluetooth) of the data transfer application as the activation item 3. For example, the activation code MnKy1-4 is associated with the use communication means “USB” cable of the data transfer application as the activation item.

  For example, the activation code MnKy2-1 is associated with the operation mode “new creation” of the mailer application as the activation item, and the activation code MnKy2-2 is the operation mode of the mailer application as the activation item 2. Corresponds to “Mail Settings”. For example, the activation code MnKy2-3 is associated with the display of the operation mode “inbox” of the mailer application as the activation item, and the activation code MnKy2-4 is the operation of the mailer application as the activation item. Corresponding to the display of the mode “transmission box”.

  For example, the activation code MnKy3-1 is associated with the browsing URL registration display function “favorite” of the web browser application as the activation item. For example, the activation code MnKy3-2 is associated with the operation mode “browser setting” of the web browser application as the activation item. For example, the activation code MnKy3-3 is associated with the operation mode selection function “mode selection” of the web browser application as the activation item. For example, the activation code MnKy3-4 is associated with the operation mode “URL input” of the web browser application as the activation item.

  FIG. 3B is a table of activation codes for the combination of the icon button at the second touch position on the touch panel display device 110 and the relative position of the first touch position with respect to the second touch position used with the table of FIG. 3A. An example is shown. Here, an icon button is a software button or software key in the form of an icon in a graphical user interface (GUI). A software button may be any display element for performing a function or operation of the software or activating an activation item by a click operation. The first and second contact positions will be specifically described later.

  In FIG. 3B, icon buttons represent identification information of various icon buttons displayed on the touch panel display device 110. The relative position is a relative positional relationship of the first contact position with respect to the second contact position when the second contact position is on the icon / button on the display screen of the touch panel display device 110. The relative position of the first contact position is, for example, any one of the azimuth range above, below, left, and right of the second contact position, any one of the distance ranges 1 to 4 from the near position to the far position from the second contact position, A combination of these azimuth ranges and distance ranges may be used. A combination of an icon button and a relative position is associated with an activation code. The user can preset which activation item is activated by a combination of which icon / button and which relative position.

  For example, a combination of an icon / button identification “IcnKy1” representing execution of an “arbitrary” application element and a relative position “above” may be associated with the activation code “MnKy1-1”. For example, a combination of the icon / button identification “IcnKy2” representing the mailer application element and the relative position “lower” may be associated with the activation code “MnKy2-2”. For example, the combination of the icon / button identification “IcnKy3” representing the web browser application element and the relative position “left” may be associated with the activation code “MnKy3-3”.

  For example, on the display screen of the touch panel display device 110, the first touch position is first above the icon button with the identification “IcnKy1” and then the second touch is on the icon button with the identification “IcnKy1” by the touch operation of the finger. Assume that a position has occurred and both positions have been confirmed. In response to the determined combination of the icon button of the second contact position and the relative position “above” of the first contact position with respect to the second contact position, the processor 102 of the information processing apparatus 10 displays the table in FIG. 3B. Based on the above, the corresponding activation code MnKy1-1 is determined. Next, the processor 102 determines an activation item corresponding to the activation code MnKy1-1 based on the table of FIG. 3A for the determined activation code MnKy1-1, and the function of the application corresponding thereto. And activate the activation item "Infrared".

  For example, on the display screen of the touch panel display device 110, the first contact position is generated below the icon button of the identification “IcnKy2”, and then the second contact position is generated on the icon button of the identification “IcnKy2”. Suppose that it is confirmed. In response to the combination of the icon button of the confirmed second contact position and the relative position “down” of the first contact position with respect to the second contact position, the processor 102 responds based on the table of FIG. 3B. The activation code MnKy2-2 to be determined is determined. Next, the processor 102 determines, for the determined activation code MnKy2-2, an activation item corresponding to the activation code MnKy2-2 based on the table of FIG. 3A, and a corresponding mailer application. Activate the operation mode "Mail setting".

  For example, on the display screen of the touch panel display device 110, the first contact position is generated on the left side of the icon button with the identification “IcnKy3”, and then the second contact position is generated on the icon button with the identification “IcnKy3”. Suppose that it is confirmed. In response to the combination of the icon button of the determined second contact position and the relative position “left” of the first contact position with respect to the second contact position, the processor 102 responds based on the table of FIG. 3B. The activation code MnKy3-3 to be determined is determined. Next, the processor 102 identifies an activation item corresponding to the activation code MnKy3-3 based on the table of FIG. 3A for the determined activation code MnKy3-3, and the corresponding web browser. Activate the operation mode "Mode selection" of the application.

  4A to 4D illustrate examples of operation procedures on the display screen of the touch panel display device 110 in the simultaneous multiple touch operation mode according to the embodiment.

4A illustrates an example of the first contact position TP1 on the display screen of the touch panel display device 110. FIG. FIG. 4B shows an example of the arrangement of the first contact position TP1 and the second contact position TP2 on the display screen of the touch panel display device 110.
In FIG. 4A, a plurality of icon buttons 202 associated with each application are displayed on the display screen by the processor 102 or its processing unit 1060 or the like.

  In FIG. 4A, the user makes a first contact (touch) by touching, for example, an index finger on the icon button 204 representing “arbitrary”, thereby generating a first contact position TP1 on the touch panel 110. In response to the occurrence of the first contact position TP1, the icon button 204 may be highlighted (highlighted) or picked up by the processor 102 on the display screen. Then, in FIG. 4B, the user makes a second contact by touching, for example, the middle finger on the icon button 206 representing the desired “mailer” to the right while holding the first contact, thereby A second contact position TP2 is generated on the touch panel 110. In response to the occurrence of the second contact position TP2, the icon 102 is highlighted (highlighted) or picked up by the processor 102.

  As the activation item display 207, an activation item “reception box” corresponding to the relative position (left) of the first contact position TP1 may be supplementarily displayed. Thereby, the user can confirm the activation item selected by the relative position. Even if the second contact position TP2 moves on the activation item display 207 after the activation item display 207 appears, the processor 102 processes the second contact position TP2 as being located on the icon button 206. Also good. Alternatively, when the second contact position TP2 moves on the activation item display 207 after the activation item display 207 appears, the processor 102 assumes that the activation item in the activation item display 207 is temporarily selected. It may be processed.

  Next, in FIG. 4B, the user holds the first contact position TP1 while activating the activation item “inbox” determined by the first contact position TP1 and the second contact position TP2. Is released and the second contact position TP2 is released. Thereby, the first contact position TP1 and the second contact position TP2 are determined. In this case, the second contact position TP2 is finally on the icon button 206 representing “mailer”, and the first contact position TP1 at that time is on the left side of the second contact position TP2. Thereafter, the first contact position TP1 may be released. As a result, the processor 102 activates the display of the mailer application operating mode “inbox” based on the tables of FIGS. 3B and 3A and displays the inbox (not shown) on the display screen. .

  As long as the contact states of both the first contact position TP1 and the second contact position TP2 are maintained during the touch operation on the touch panel 110, the first contact position TP1 and the second contact position TP2 are not fixed and can be changed. . It is allowed to change the first contact position TP1 and the second contact position TP2 by moving, for example, two fingers in a contact state on the touch panel 110. Further, even when the second contact position TP2 is moved to a non-icon button area that is neither on the icon button nor on the activation item display 207 and released, the second contact position TP2 is not fixed. On the other hand, when the first contact position TP1 is released while the second contact position TP2 is maintained, the processor 102 changes or updates the second contact position TP2 to the first contact position TP1, and then a new second contact position TP1. You may wait for generation | occurrence | production of TP2. Thereby, the user can fix and confirm the first contact position TP1 and the second contact position TP2 without error or when necessary.

FIG. 4C shows another example of the first contact position TP1 on the display screen of the touch panel display device 110. FIG. 4D shows another example of the arrangement of the first contact position TP1 and the second contact position TP2 on the display screen of the touch panel display device 110.
In FIG. 4C, a plurality of icon buttons 202 associated with each application are displayed on the display screen by the processor 102 or its processing unit 1060.

  In FIG. 4C, the user touches, for example, the middle finger on the non-icon button area on the display screen to form the first contact (touch), and the first contact position TP1 is generated. Such a touch position on the non-icon button area may be picked up and displayed by the processor 102 with, for example, a circular symbol 208 on the display screen. The circular symbol 208 moves as indicated by the dashed arrow with the movement of the contact position. Then, in FIG. 4D, the user makes a second contact, for example by touching an index finger 204 on the icon button 204 representing the desired “arbitrary” underneath, maintaining the first contact, and the second contact. A position TP2 is generated. In response, icon button 204 may be highlighted or picked up. In this case, for example, the corresponding activation item “infrared” (upper) may be displayed as the activation item display 207 in an auxiliary manner.

  Next, in FIG. 4D, the user holds the first contact position TP1 while activating the activation item “infrared” determined by the first contact position TP1 and the second contact position TP2, The contact is released and the second contact position TP2 is released. Thereby, the first contact position TP1 and the second contact position TP2 are determined. In this case, the second contact position TP2 is finally on the icon button 204 representing “arbitrary”, and the first contact position TP1 at that time is located above the second contact position TP2. As a result, the processor 102 activates the data transfer application and its communication medium “infrared” transceiver based on the tables of FIGS. 3B and 3A.

  Thus, in the simultaneous multiple touch operation mode, the user can perform a plurality of touch operations using two fingers. In addition, the information processing apparatus 10 performs the operation on the icon button 206 or 204 while the first contact position TP1 is maintained after the first contact position TP1 is formed at an arbitrary position on the display screen. When the second contact is formed, the multi-touch operation mode is determined. Thereby, the information processing apparatus 10 can distinguish the case where the icon button 206 or 204 is operated in the multi-touch operation mode from the case where the icon button 206 or 204 is operated in the single touch operation mode.

  5A and 5B show an example of an operation procedure on the display screen of the touch panel display device 110 in the time difference multiple touch operation mode using the timer 1128 according to the embodiment.

  FIG. 5A shows an example of the arrangement of the first contact position TP1 and the second contact position TP2 on the display screen of the touch panel display device 110.

  In FIG. 5A, the user first makes a first contact (touch) by touching the icon button 204 representing “arbitrary” to generate a first contact position TP1. Thereby, the icon button 204 may be highlighted or picked up by the processor 102. Next, the user releases the first contact position TP1 by releasing the first contact. In this case, the highlighted display of the icon button 204 may be maintained. Within a time limit set in the timer 1128 (for example, 0.5 second or 1 second) after the release of the first contact position TP1, the user moves the icon button 206 representing the desired “mailer” on the right side of the timer 1128. A finger is brought into contact with the second contact to form a second contact, and the second contact position TP2 is generated. In response, icon button 206 may be highlighted or picked up.

  Then, in order to activate the activation item determined by the first contact position TP1 and the second contact position TP2, the user releases the second contact and releases the second contact position TP2. Thereby, the first contact position TP1 and the second contact position TP2 are determined. In this case, the second contact position TP2 is finally on the icon button 206 representing “mailer”, and the position of the first contact position TP1 at that time is on the left side of the second contact position TP2. As a result, the processor 102 activates the display of the mailer application operating mode “inbox” based on the tables of FIGS. 3B and 3A, as in FIG. 4B, and an inbox (not shown). Is displayed on the display screen.

  When the second contact position TP2 occurs after the generation and disappearance of the first contact position TP1, the activation item display 207 is displayed by the processor 102 in the vicinity of the icon button 206 or in the adjacent region as in FIG. 4B. May be displayed in an auxiliary manner.

  The first contact position TP1 and the second contact position TP2 can be changed by moving the finger in a contact state on the display screen. When the second contact position TP2 is moved to the non-icon / button region and released within the time limit, the second contact position TP2 may be released without being fixed. When the second contact position TP2 is moved to the non-icon / button region and released beyond the time limit, the first contact position TP1 and the second contact position TP2 may not be determined or may be released together. Thereby, the user can fix and confirm and confirm the first contact position TP1 and the second contact position TP2 without error or when necessary.

  FIG. 5B shows another example of the arrangement of the first contact position TP1 and the second contact position TP2 on the display screen of the touch panel display device 110.

  In FIG. 5B, the user first makes a first contact (touch) by bringing a finger into contact with the non-icon button area on the display screen, and the first contact position TP1 is generated. In response, such contact location on the non-icon button area may be picked up (208). Next, the user releases the first contact and releases the first contact position TP1. In this case, the pickup display of the first contact position TP1 may be maintained. Within the time limit of the timer 1128 (for example, 0.5 second or 1 second) after the release of the first contact position TP1, the user places his finger on the icon button 204 representing the desired “any” below it. A second contact is formed by contact, and a second contact position TP2 is generated. In response, icon button 204 may be highlighted or picked up. Also in this case, as in the case of FIG. 5A, the activation item display 207 may be supplementarily displayed.

  Next, the user releases the second contact and releases the second contact position TP2 in order to activate the activation item determined by the first contact position TP1 and the second contact position TP2. Thereby, the first contact position TP1 and the second contact position TP2 are determined. In this case, the second contact position TP2 is finally on the icon button 204 representing “arbitrary”, and the position of the first contact position TP1 at that time is above the second contact position TP2. As a result, the processor 102 activates the transceiver of the data transfer application and its used communication medium “infrared” based on the tables of FIGS. 3B and 3A, as in FIG. 4D.

  In the time difference multiple touch operation mode using a timer, the user can perform a plurality of touch operations in order using one finger, a touch pen, or other touch means. Further, the processor 102 determines that the second contact is formed on the icon button 206 or 204 within a certain time limit after the first contact position TP1 is formed at an arbitrary position and disappears on the display screen. In addition, the multi-touch operation mode is determined. Thereby, the information processing apparatus 10 can distinguish the case where the icon button 206 or 204 is operated in the multi-touch operation mode from the case where the icon button 206 or 204 is operated in the single touch operation mode.

  6A to 6D illustrate an example of an operation procedure on the display screen of the touch panel display device 110 in another time difference multiple touch operation mode that does not use the timer 1128 according to the embodiment.

  6A and 6B show an example of the arrangement of the icon / button 210 for switching the timer mode on the display screen of the touch panel display device 110. FIG.

  In FIG. 6A, when the finger is touched on the icon button 210, the icon button 210 is highlighted, and when the touch is released, the time difference multiple touch operation mode of the information processing apparatus 10 is changed to the timer on mode of FIGS. 5A and 5B. To timer off mode. As a result, the timer 1128 is turned off or its measurement time is set to infinity. As shown in FIG. 6B, in the timer off mode, the processor 102 may display, for example, a changed background color and / or a message representing the timer off mode on the display screen.

  FIG. 6C shows an example of the arrangement of the first contact position TP1 and the second contact position TP2 on the display screen of the touch panel display device 110 in the timer-off mode of FIG. 6B.

  6C, the information processing apparatus 10 operates in the same manner as in FIG. 5A except that the timer 1128 is turned off or the setting time limit of the timer 1128 is infinite. In this case, after forming and releasing the first contact position TP1 by making the first contact, the user touches the icon button 206 representing the desired “mailer” on the right side at any point in time. A second contact is formed, and a second contact position TP2 is generated.

  FIG. 6D shows another example of the arrangement of the first contact position TP1 and the second contact position TP2 on the display screen of the touch panel display device 110 in the timer-off mode of FIG. 6B.

  6D, the information processing apparatus 10 operates in the same manner as in FIG. 5B except that the timer 1128 is turned off or the set time of the timer 1128 becomes infinite. Again, after the user has formed and released the first contact position TP1, the user can place the first contact on the icon button 204 representing the desired “arbitrary” below it at any point in time. The second contact position TP2 is generated.

  Even in the time difference multiple touch operation mode without using a timer, the user can perform a plurality of touch operations using one finger, a touch pen, or other touch means. In addition, the information processing apparatus 10 may be displayed on the icon button 206 or 204 at any time after the timer off mode is set on the display screen and the first contact position TP1 is formed at an arbitrary position and disappears. When the second contact position TP2 is formed, the multi-touch operation mode is determined. Thereby, the information processing apparatus 10 can distinguish the case where the icon button 206 or 204 is operated in the multi-touch operation mode from the case where the icon button 206 or 204 is operated in the single touch operation mode.

  FIG. 7A shows an example of a table of one or more activation items for each activation code.

  In FIG. 7A, each activation code may activate a display operation of options representing one or more activation items. The table of FIG. 7A is stored in a storage device such as the memory 104 or the storage device 130, for example.

  For example, the activation code MnKy1-1 is displayed as the activation items 1, 3, and 4 as the use communication medium “infrared” of the data transfer application, the use communication means “Bluetooth (registered trademark)” (Bluetooth), and the Corresponding to the used communication means “USB” cable.

  For example, the activation code MnKy2-3 corresponds to the display of the operation mode “new creation”, “mail setting”, “inbox” and “outbox” of the mailer application as the activation items 1 to 4 to be displayed. It is attached.

  For example, the activation code MnKy3-2 includes the URL registration display function “favorite”, the operation mode “browser setting”, the operation mode selection function “mode selection”, and the activation items 1 to 4 to be displayed. The operation mode is associated with “URL input”.

  FIG. 7B shows an icon / button at the second contact position on the touch panel display device 110 and the relative position of the first contact position with respect to the second contact position, which is used together with the table of FIG. An example of an activation code table for a combination is shown. The table of FIG. 7B is stored in a storage device such as the memory 104 or the storage device 130, for example.

  In FIG. 7B, the user can preset which one or more activation item display operations are activated by a combination of which icon button and which relative position.

  For example, on the display screen of the touch panel display device 110, it is assumed that the first contact position first occurs on the left of the icon button with the identification “IcnKy2”, and then the second contact position occurs on the icon button with the identification “IcnKy2”. In response to the combination of the second touch position icon button and the first touch position relative position “left” with respect to the second touch position, processor 102 activates the corresponding activation based on the table of FIG. 7B. The code MnKy2-3 is determined. Next, the processor 102 determines activation items 1 to 4 corresponding to the activation code MnKy2-3 based on the table of FIG. 7A for the determined activation code MnKy2-3. The processor 102 displays the determined activation items “New”, “Mail Settings”, “Inbox”, and “Outbox” as options around the icon button. The user moves the second contact position onto the display of the desired activation item, e.g. "Send Box" and releases it there to select or determine the activation item. The processor 102 activates the corresponding mailer application function and its selected activation item “outbox”.

  For example, on the display screen of the touch panel display device 110, it is assumed that the first contact position is first generated below the icon button having the identification “IcnKy3”, and then the second contact position is generated on the icon button having the identification “IcnKy3”. In response to the combination of the second touch position icon button and the first touch position relative position “down” with respect to the second touch position, processor 102 activates the corresponding activation based on the table of FIG. 7B. The code MnKy3-2 is determined. Next, the processor 102 determines activation items 1 to 4 corresponding to the activation code MnKy3-2 based on the table of FIG. 7A for the determined activation code MnKy3-2. The processor 102 displays the determined activation items “favorites”, “browser settings”, “mode selection”, and “URL input” as options around the icon button. The user moves the second contact position onto the display of a desired activation item, for example “URL input”, and releases it there to select or determine the activation item. The processor 102 activates the function of the corresponding web browser application and its selected activation item “URL entry”.

  8A to 8D show an example of an operation procedure on the display screen of the touch panel display device 110 in the simultaneous multiple touch operation mode when selectable activation items are displayed according to the embodiment.

  FIG. 8A shows an example of the first contact position TP <b> 1 on the display screen of the touch panel display device 110. FIG. 8B shows an example of the arrangement of the first contact position TP1 and the second contact position TP2 on the display screen of the touch panel display device 110. FIGS. 8A and 8B are similar to FIGS. 4A and 4B, except for the activation item display 207 and the activation item activation form.

  When the first contact position TP1 and the second contact position TP2 on the icon button 206 occur, the processor 102 activates the activation item 1 associated with the icon button 206 near or adjacent to the icon button 206. The list or menu of ˜4 is displayed as the activation item display 207. As a result, the activation items 1 to 4 are displayed in the vicinity of the icon button 206 as options determined according to the relative positions. In this case, as the activation item display 207, the corresponding activation items “new creation” (top), “mail setting” (bottom), “inbox” (left), “outbox” (right) in the table of FIG. ) Is displayed.

  Next, in FIG. 7B, in order to activate the desired activation item “new creation”, the user holds the first contact position TP1 and displays the second contact on the display of the activation item “new creation”. And then released there to release the second contact position TP2. Thereby, the second contact position TP2 is determined. Thereafter, the first contact position TP1 may be released. As a result, the processor 102 activates display of the operation mode “new creation” of the mailer application, and a new creation screen (not shown) is displayed on the display screen.

  As long as the contact state of both the first contact position TP1 and the second contact position TP2 is maintained on the touch panel 110, the first contact position TP1 and the second contact position TP2 are not determined and can be changed. When the second contact position TP2 is released on the non-icon button area or icon button that is not the activation item display 207, the second contact position TP2 may not be determined. As an alternative, when one icon button is associated with only one activation item, the second contact position TP2 is released on the icon button or activation item display 207. The second contact position TP2 may be confirmed and the activation item may be activated. On the other hand, when the first contact position TP1 is released while the second contact position TP2 is maintained, the processor 102 changes or updates the second contact position TP2 to the first contact position TP1, and then a new second contact position TP1. You may wait for generation | occurrence | production of TP2. Thereby, the user can fix and confirm the first contact position TP1 and the second contact position TP2 without error or when necessary.

  8C and 8D are similar to FIGS. 4C and 4D except for the activation item display 207 and the activation item activation form. In this case, for example, “infrared” (upper), “Bluetooth (registered trademark)” (left), “USB” (right), and “infrared” (upper) are displayed as the activation item display 207.

  Next, in FIG. 8D, in order to activate the desired activation item “infrared”, the user moves the second contact over the display of the activation item “infrared” while holding the first contact position TP1. Then, the second contact position TP2 is released by releasing it there. Thereby, the second contact position TP2 is determined. As a result, the processor 102 activates the data transfer application and the transceiver used for its communication medium “infrared”.

  On the display screen, when the first contact position TP1 is sequentially slid with respect to the second contact position TP2 on the same button and the relative position thereof is changed, different activation item lists are activated. Can be sequentially displayed as the generalized item display 207. For example, referring to FIGS. 7B and 7A, if the relative position of the first contact position TP1 is moved up-left-down with respect to the icon button IcnKy3 at the second contact position TP2, the icon button is moved around the icon button. The following three lists of activation items are sequentially changed and displayed. In this case, the icon button IcnKy3 represents a web browser. Initially, the activation code MnKy3-1 is selected for the relative position “upper”, and the four activation items “top” (upper), “browser history” (lower), “other settings” ( A list of “search” (right side) and “search” (right side) is displayed as the activation item display 207. The activation code MnKy3-3 is selected for the next relative position “left”, and the four activation items “Kansai road information” (upper side) and “Kansai Railway” are used as search keywords in the web browser search function. A list of “information” (lower side), “Nadeshiko Japan” (left side), and “Kansai Museum of Art” (right side) is displayed as the activation item display 207. For the next relative position “down”, the activation code MnKy3-2 is selected and the four activation items “favorite” (upper), “browser settings” (lower), “mode selection” (left) , A list of “URL input” (right side) is displayed as the activation item display 207. In this case, the user can select and activate a desired one out of a total of 12 activation item options.

  9A and 9B show an example of an operation procedure on the display screen of the touch panel display device 110 in the time difference multiple touch operation mode using the timer 1128 when a selectable activation item is displayed according to the embodiment. Yes.

  FIG. 9A shows an example of the arrangement of the first contact position TP1 and the second contact position TP2 on the display screen of the touch panel display device 110.

  FIG. 9A is similar to FIG. 5A except for the activation item display 207 and the activation item activation form. In this case, activation items 1 to 4 are displayed in the vicinity of the icon button 206 as options determined according to the relative position. In this case, as the activation item display 207, the corresponding activation items “new creation” (top), “mail setting” (bottom), “inbox” (left), “outbox” (right) in the table of FIG. ) Is displayed.

  Next, the user moves the second contact onto the display of the desired activation item “new creation” and releases it to release the second contact position TP2. Thereby, the second contact position TP2 is determined. As a result, as in the case of FIG. 8B, the processor 102 activates the display of the operation mode “new creation” of the mailer application, and displays a new creation screen (not shown) on the display screen.

  FIG. 9B shows another example of the arrangement of the first contact position TP1 and the second contact position TP2 on the display screen of the touch panel display device 110.

  FIG. 9B is the same as FIG. 5B except for the activation item display 207 and the activation item activation form. Also in this case, the activation item display 207 is displayed as in the case of FIG. 9A.

  The user then moves the second contact over the display of the desired activation item “infrared” and releases it there to release the second contact position TP2. Thereby, the second contact position TP2 is determined. As a result, the processor 102 activates the transceiver of the data transfer application and its communication medium “infrared” as in FIG. 8D.

  10A to 10D are examples of operation procedures on the display screen of the touch panel display device 110 in another time difference multiple touch operation mode in which the timer 1128 is not used when selectable activation items are displayed according to the embodiment. Show.

  FIGS. 10A and 10B show an example of the arrangement of the icon buttons 210 for switching the timer mode on the display screen of the touch panel display device 110. 10A and 10B are similar to FIGS. 6A and 6B.

  FIG. 10C shows an example of the arrangement of the first contact position TP1 and the second contact position TP2 on the display screen of the touch panel display device 110 in the timer-off mode of FIG. 10B.

  10C, the information processing apparatus 10 operates in the same manner as in FIG. 9A except that the timer 1128 is turned off or the setting time limit of the timer 1128 is infinite. In this case, after forming and releasing the first contact position TP1 by making the first contact, the user touches the icon button 206 representing the desired “mailer” on the right side at any point in time. A second contact is formed, and the second contact position TP2 is generated.

  FIG. 10D shows another example of the arrangement of the first contact position TP1 and the second contact position TP2 on the display screen of the touch panel display device 110 in the timer-off mode of FIG. 10B.

  10D, the information processing apparatus 10 operates in the same manner as in FIG. 9B except that the timer 1128 is turned off or the set time of the timer 1128 becomes infinite. Again, after the user has formed and released the first contact position TP1, the user can place the first contact on the icon button 204 representing the desired “arbitrary” below it at any point in time. The second contact position TP2 is generated.

  FIG. 11A shows an example in which the positional relationship of the first contact position TP1 with respect to the second contact position TP2 is classified into four azimuth ranges. In this case, the positional relationship between the first contact position TP1 and the second contact position TP2 is, for example, four directions on the top, bottom, left, and right with two straight lines intersecting at an angle of 45 degrees and 135 degrees at the second contact position TP2. Classified into a range. The functions or data representing the two straight lines may be stored in the memory 104 or the storage device 130, or may be incorporated as part of the application. The two straight lines may be displayed as auxiliary lines 220 by the processor 102.

  In FIG. 11A, the processor 102 compares the coordinates (x1, y1) of the first contact position TP1 with the coordinates (x2, y2) of the second contact position TP2, so that the first contact position TP1 becomes the second contact position TP2. Which of the four azimuth ranges of up, down, left and right may be determined. For example, the processor 102 determines that the relative difference is obtained when the difference (Δx, Δy) = (x1−x2, y1−y2) between two contact positions TP1 and TP2 satisfies Δy ≧ 0 and | Δy / Δx | ≧ 1. The position may be determined as “up”. For example, the processor 102 may determine that the relative position is “down” when the coordinate difference (Δx, Δy) satisfies Δy <0 and | Δy / Δx | ≧ 1. For example, the processor 102 may determine that the relative position is “right” when the coordinate difference (Δx, Δy) satisfies Δx ≧ 0 and | Δy / Δx | <1. For example, the processor 102 may determine that the relative position is “left” when the difference (Δx, Δy) between the coordinates is Δx <0 and | Δy / Δx | <1.

  FIG. 11B shows an example in which the positional relationship of the first contact position TP1 with respect to the second contact position TP2 is classified into four distance ranges. In this case, the positional relationship between the first contact position TP1 and the second contact position TP2 is, for example, 4 of the first distance to the fourth distance with three concentric circles having different radii centered on the second contact position TP2 as boundaries. Into one distance range. Functions or data representing the three concentric circles may be stored in the memory 104 or storage device 130, or may be incorporated as part of the application. The three circles may be displayed as auxiliary lines 222 by the processor 102.

In FIG. 11B, for example, the processor 102 sets the relative position of the closest difference when the coordinate difference (Δx, Δy) satisfies, for example, the distance D = (Δx ² + Δy ² ) ^1/2 ≦ 2 cm or 100 pixels. You may determine with 1 distance range. For example, for the coordinate difference (Δx, Δy), the processor 102 may determine the relative position as the second distance range when the distance D> 2 cm or 100 pixels and D ≦ 3 cm or 150 pixels are satisfied. For example, for the coordinate difference (Δx, Δy), the processor 102 may determine the relative position as the third distance range when the distance D> 3 cm or 150 pixels and D ≦ 4 cm or 200 pixels are satisfied. For example, the processor 102 may determine the relative position as the fourth distance range when the distance D> 4 cm or 200 pixels is satisfied for the coordinate difference (Δx, Δy).

  FIG. 11C shows an example in which the positional relationship of the first contact position TP1 with respect to the second contact position TP2 is classified into eight azimuth ranges. In this case, the positional relationship between the first contact position TP1 and the second contact position TP2 is, for example, 8 boundaries with four straight lines intersecting at angles of 0 degrees, 45 degrees, 90 degrees, and 135 degrees at the second contact position TP2. It is classified into an azimuth range. The functions or data representing the four straight lines may be stored in the memory 104 or the storage device 130, or may be incorporated as part of the application. In this case, the eight azimuth ranges may be identified as the upper right side, the upper left side, the lower right side, the lower left side, the upper left side, the lower left side, the upper right side, and the lower right side. The four straight lines may be displayed as auxiliary lines 224 by the processor 102.

  In FIG. 11C, for example, the processor 102 compares the relative difference when the coordinate difference (Δx, Δy) = (x1−x2, y1−y2) satisfies Δx ≧ 0, Δy ≧ 0, and | Δy / Δx | ≧ 1. The position may be determined as “upper right”. For example, the processor 102 may determine that the relative position is “upper left” when the coordinate difference (Δx, Δy) satisfies Δx <0, Δy ≧ 0 and | Δy / Δx | ≧ 1. For example, the processor 102 may determine the relative position as “lower right” when the coordinate difference (Δx, Δy) satisfies Δx ≧ 0, Δy <0 and | Δy / Δx | ≧ 1. For example, the processor 102 may determine the relative position as “lower left side” when the coordinate difference (Δx, Δy) satisfies Δx <0, Δy <0 and | Δy / Δx | ≧ 1. For example, the processor 102 may determine that the relative position is “upper right” when the coordinate difference (Δx, Δy) satisfies Δx ≧ 0, Δy ≧ 0, and | Δy / Δx | <1. For example, the processor 102 may determine that the relative position is “lower right” when the coordinate difference (Δx, Δy) satisfies Δx ≧ 0, Δy <0 and | Δy / Δx | <1. . For example, the processor 102 may determine that the relative position is “upper left” when the coordinate difference (Δx, Δy) satisfies Δx <0, Δy ≧ 0 and | Δy / Δx | <1. For example, when the coordinate difference (Δx, Δy) satisfies Δx <0, Δy <0 and | Δy / Δx | <1, the processor 102 may determine the relative position as “lower left”. .

  FIG. 11D shows an example in which the positional relationship of the first contact position TP1 with respect to the second contact position TP2 is classified into a range of combinations of four directions and four distances. In this case, the positional relationship is defined by four azimuth ranges with two straight lines intersecting at an angle of 45 degrees and 135 degrees in FIG. 11A as boundaries, and three concentric circles having different radii in FIG. 11B as boundaries. By combining with the four distance ranges from the first distance to the fourth distance, a total of 16 ranges are classified. The functions or data representing the two straight lines and the three circles may be stored in the memory 104 or the storage device 130, or may be incorporated as part of the application. Two straight lines and three circles representing the boundary line may be displayed as auxiliary lines 226 by the processor 102.

  The display of the auxiliary lines 200 to 226 allows the user to check the correctness of the relative position of the first contact position TP1 for selecting a desired activation item. Further, the display of the auxiliary lines 200 to 226 allows the user to accurately form the relative position of the first contact position TP1 for selecting a desired activation item even when the number of available relative positions is large. Can do.

  12A to 12C show an example of an operation procedure for activating a desired activation item in the information processing apparatus 10 in the single touch operation mode.

  Referring to FIG. 12A, the user activates the web browser application by touching and releasing (clicking) the finger on the “browser” icon button 205 on the display screen of the touch panel display device 110. Then, the processor 102 displays a list or menu of browser activation items as a browser screen as shown in FIG. 12B. In the display screen of FIG. 12B, the user activates the activation item “other setting” of the web browser application by touching and releasing the finger on the button of the activation item “other setting”. Then, the processor 102 displays a list of the next lower activation items “other setting 1” to “other setting 5” as a browser screen as shown in FIG. 12C. The user activates the desired activity item by selecting the desired activation item from the list of FIG. 12C by touching and releasing the button with the finger.

  13A and 13B show an example of an operation procedure for activating a desired activation item in the information processing apparatus 10 in the multi-touch operation mode. The multiple touch operation mode may be any of the above-described simultaneous multiple touch operation mode, a time difference multiple touch operation mode using the timer 1128, and a time difference multiple touch operation mode not using the timer 1128.

  FIG. 14A shows an example of a table of activation items for each activation code as applied to FIGS. 13A and 13B. FIG. 14B shows the activation code associated with the combination of the icon button at the second contact position TP2 and the relative position of the first contact position TP1 with respect to the second contact position TP2 applied to FIGS. 13A and 13B. An example of the table is shown.

  In FIG. 14A, for example, the activation code MnKy3-1 is associated with the activation item “other setting” of the web browser application, and the activation code MnKy3-2 is associated with the activation item “favorite” of the web browser application. It is associated. For example, the activation code MnKy3-3 is associated with the activation item “browser history” of the web browser application.

  In FIG. 14B, for example, a combination of an icon / button identification “IcnKy3” representing activation of the web browser application and a relative position “above” is associated with the activation code “MnKy3-1”. For example, a combination of an icon / button identification “IcnKy3” indicating activation of the web browser application and a relative position “lower” is associated with the activation code “MnKy3-2”. For example, the combination of the icon / button identification “IcnKy3” representing the activation of the web browser application and the relative position “right” is associated with the activation code “MnKy3-3”.

  Referring to FIG. 13A, on the display screen of touch panel display device 110, the user touches his / her finger at a position that is in a relative position with respect to icon button 205 for activation of a desired “browser” application. Thus, the first contact position TP1 is generated. Next, the user holds the first contact position TP1 in the simultaneous multiple touch operation mode, cancels the first contact position TP1 in the time difference multiple touch operation mode, and makes the finger touch the icon button 205 of the “browser”. Then, the second contact position TP2 is generated. Next, the user releases the finger to release the second contact position TP2. Thereby, the identification information of the icon / button at the second contact position TP2 and the relative position of the first contact position TP1 with respect to the second contact position TP2 are determined. When the second contact position TP2 occurs, the processor 102 displays the activation item associated with the icon button 205 in the relationship with the relative position around the icon button 205 as described above. 207 may be displayed. In this case, even when the second contact position TP2 is moved on the activation item display 207 and released, the activation item may be selected and activated.

  The processor 102 responds according to the identification of the icon button 205 at the second contact position TP2 and the relative position of the first contact position TP1, as shown in FIG. 13B, based on the tables of FIGS. 14B and 14A. Activate the display of the activation item "Other settings". FIG. 13B corresponds to FIG. 12C. As a result, a list of lower-level activation items “other setting 1” to “other setting 5” is displayed as the web browser application screen. That is, the intermediate activation item or display screen in FIG. 12B is skipped, and the desired activation item or display screen in FIG. 13B is displayed after the screen in FIG. 13A.

  As described above, in the touch panel display device 110, the operation procedure and operation time for activating a desired activation item such as a desired operation mode or information display of the web browser can be reduced.

  15A to 15C show another example of an operation procedure for activating a desired activation item in the information processing apparatus 10 in the single touch operation mode.

  Referring to FIG. 15A, the user activates the schedule application by touching and releasing the finger on the “schedule” icon button 205 on the display screen of the touch panel display device 110. As a result, as shown in FIG. 15B, a list of schedule activation items is displayed as a schedule screen. In the display screen of FIG. 15B, the user activates the operation mode “schedule of work” of the schedule application by touching and releasing the finger on the button of the activation item “schedule of work”. As a result, as shown in FIG. 15C, a “work schedule” display screen is displayed as the schedule screen.

  16A to 16E are modifications of the operation procedure of FIGS. 15A to 15C, and show another example of the operation procedure for activating a desired activation item in the information processing apparatus 10 in the single touch operation mode. Yes.

  Referring to FIG. 16A, on the display screen of touch panel display device 110, the user activates the unlocking process by touching and releasing the finger on the “setting” icon / button. In FIG. 16B, the user enters a password in the unlock password input field on the display screen, and touches and releases the “setting” icon / button to release the “secret plan” display suppression. To do. In FIG. 16C, the user activates the schedule application by touching and releasing the finger on the “schedule” icon button 205 on the display screen. Thereby, as shown in FIG. 16D, a list of schedule activation items is displayed as a schedule screen. In this case, an item “secret schedule” not displayed in FIG. 15B is added and displayed as a list of schedule activation items. The user activates the operation mode “secret schedule” of the schedule application by touching and releasing the finger on the button of the activation item “secret schedule” on the display screen. Thereby, as shown in FIG. 16E, a display screen of “secret schedule” is displayed as the schedule screen.

  17A and 17B show an example of an operation procedure for activating a desired activation item in the information processing apparatus 10 in the multiple touch operation mode.

  18A to 18D are modifications of the operation procedure of FIGS. 17A and 17B, and show another example of the operation procedure for activating a desired activation item in the information processing apparatus 10 in the multi-touch operation mode. .

  FIG. 19A shows an example of a table of activation items for each activation code as applied to FIGS. 17A, 17B and 18A-18D. 19B is associated with the combination of the icon button at the second contact position TP2 and the relative position of the first contact position TP1 with respect to the second contact position TP2, as applied to FIGS. 17A, 17B, and 18A to 18D. An example of the activation code table is shown.

  In FIG. 19A, for example, the activation code MnKy3-1 is associated with the activation item “schedule of work” of the schedule application, and the activation code MnKy3-2 is the activation item of the schedule application “private schedule”. Is associated with. For example, the activation code MnKy3-3 is associated with the activation item “secret schedule” of the schedule application.

  In FIG. 19B, for example, a combination of an icon / button identification “IcnKy3” representing activation of the schedule application and a relative position “above” is associated with the activation code “MnKy3-1”. For example, a combination of the icon / button identification “IcnKy3” representing activation of the schedule application and the relative position “lower” is associated with the activation code “MnKy3-2”. For example, a combination of an icon / button identification “IcnKy3” representing activation of the schedule application and a relative position “right” is associated with the activation code “MnKy3-3”.

  Referring to FIG. 17A, on the display screen of touch panel display device 110, the user touches the icon button 205 for activating the desired “schedule” with a finger at a position that is in a relative positional relationship. A first contact position TP1 is generated. Next, the user releases the first contact position TP1 in the time difference multiple touch operation mode while keeping the first contact position TP1 in the simultaneous multiple touch operation mode, and touches the “schedule” icon button 205 with the finger. Then, the second contact position TP2 is generated. Next, the user releases the finger to release the second contact position TP2. Thereby, the identification information of the icon / button at the second contact position TP2 and the relative position of the first contact position TP1 with respect to the second contact position TP2 are determined. When the second contact position TP2 occurs, the processor 102 displays the activation item associated with the icon button 205 in the relationship with the relative position around the icon button 205 as described above. 207 may be displayed.

  The processor 102 responds according to the identification of the icon button 205 at the second contact position TP2 and the relative position of the first contact position TP1, as shown in FIG. 17B, based on the tables of FIGS. 19B and 19A. Activate the activation item “Schedule of work”. FIG. 17B corresponds to FIG. 15C. As a result, a “work schedule” display screen is displayed as the schedule screen. That is, the intermediate activation item or display screen in FIG. 15B is skipped, and the desired activation item or display screen in FIG. 17B is displayed.

  As described above, in the touch panel display device 110, an operation procedure and an operation time for activating a desired activation item such as a schedule display of a desired category of the schedule application can be reduced.

  Referring to FIG. 18A, on the display screen of touch panel display device 110, the user touches the icon button 205 for activating the desired “schedule” application with a position in a relative positional relationship. The first contact position TP1 is generated. Next, as described above, the user keeps the first contact position TP1 or cancels the first contact position TP1, and touches the “schedule” icon button 205 with the finger, so that the second contact position TP2 is reached. Then, the finger is released to release the second contact position TP2. Thereby, the identification information of the icon / button at the second contact position TP2 and the relative position of the first contact position TP1 with respect to the second contact position TP2 are determined.

  The processor 102 is shown in FIG. 18D corresponding to FIG. 16E based on the tables of FIGS. 19B and 19A according to the identification of the icon button 205 of the second contact position TP2 and the relative position of the first contact position TP1. Attempt to activate the corresponding activation item “secret schedule”. However, in order to activate the activation item “secret schedule”, display suppression of “secret schedule” needs to be cancelled. Therefore, in this case, the processor 102 activates the unlocking process and displays the display screen of FIG. 18B before that. In FIG. 18B, the user inputs a password in the unlock password input field on the display screen, and touches and releases the icon button “setting” to release the display suppression of “secret schedule”. .

  Then, the processor 102 immediately displays the “secret schedule” display screen of FIG. 18D corresponding to FIG. 16E or displays the display screen of FIG. 18C corresponding to FIG. 16D. When the display screen of FIG. 18C is displayed, the user touches and releases the desired activation item “secret schedule” button on the display screen to release the activation item “secret” of the schedule application. Activate the “plan” display. Then, as shown in FIG. 18D, a display screen of “secret schedule” is displayed as a schedule screen.

  As described above, in the touch panel display device 110, the operation procedure and operation time for activating a desired activation item such as a schedule display of a category that requires unlocking of the schedule application can be greatly reduced. it can.

  FIG. 20A shows another example of a table of activation items for each activation code. FIG. 20B shows another example of the activation code table associated with the combination of the icon button at the second contact position TP2 and the relative position of the first contact position TP1 with respect to the second contact position TP2. Yes.

  In FIG. 20A, for example, the activation code MnKy3-1 is associated with the activation item “work mode” of the mailer application, and the activation code MnKy3-2 is associated with the activation item “private mode” of the mailer application. It is associated. For example, the activation code MnKy3-3 is associated with the activation item “secret mode” of the mailer application.

  In FIG. 20B, for example, a combination of an icon / button identification “IcnKy3” representing activation of the mailer application and a relative position “above” is associated with the activation code “MnKy3-1”. For example, a combination of the icon / button identification “IcnKy3” representing activation of the mailer application and the relative position “lower” is associated with the activation code “MnKy3-2”. For example, a combination of the icon / button identification “IcnKy3” representing activation of the mailer application and the relative position “right” is associated with the activation code “MnKy3-3”.

  For example, on the display screen of the touch panel display device 110, the user generates a first contact position TP1 by touching a region above an icon / button for activating a desired “mailer”. Next, as described above, the user holds the first contact position TP1 or cancels the first contact position TP1, and touches the icon button representing the “mailer” application with the finger, so that the second contact position is reached. TP2 is generated, and then the contact is released to release the second contact position TP2. According to the operation, the processor 102 activates the corresponding activation item “work mode” based on the tables of FIGS. 20B and 20A, and displays a display screen of “work mode” as a mailer screen.

  As described above, in the touch panel display device 110, it is possible to reduce an operation procedure and an operation time for activating a desired activation item such as a mailer's desired operation mode.

  Further, for example, on the display screen of the touch panel display device 110, the user generates a first contact position TP1 by bringing a finger into contact with an area to the right of an icon / button for activating a desired “mailer”. Next, as described above, the user holds the first contact position TP1 or cancels the first contact position TP1, and touches the icon button representing the “mailer” application with the finger, so that the second contact position is reached. TP2 is generated, and then the second touch position TP2 is released by releasing the touch. In accordance with that operation, processor 102 attempts to activate the corresponding activation item “secret mode” based on the tables of FIGS. 20B and 20A. However, in order to activate the activation item “secret mode”, display suppression of “secret mode” needs to be released. Therefore, in this case, before that, the processor 102 activates the unlocking process and displays a display screen for unlocking similar to FIG. 18B. On the unlock display screen, the user inputs a password in the unlock password input field to cancel the display suppression of “secret mode”. Next, the processor 102 immediately displays the display screen of “secret mode” or displays the display screen of the list of three modes. If a display screen of a list of three modes is displayed, the user touches and releases the button on the activation item “secret mode” on the display screen to release the “secret mode” of the mailer application. Activate the display of. As a result, the “secret mode” display screen is displayed as the mailer screen.

  Thus, in the touch panel display device 110, the operation procedure and operation time for activating a desired activation item such as an operation mode that requires unlocking of the mailer can be greatly reduced.

  FIG. 21A shows another example of a table of activation items for each activation code. FIG. 21B shows another example of the activation code table associated with the combination of the icon button at the second contact position TP2 and the relative position of the first contact position TP1 with respect to the second contact position TP2. Yes.

  In FIG. 21A, for example, the activation code MnKy3-1 is associated with the activation item “page of YH” of the web browser application, and the activation code MnKy3-2 is the activation item “NK newspaper of the web browser application”. Is associated with the “page”. For example, the activation code MnKy3-3 is associated with the activation item “blog page” of the web browser application.

  In FIG. 21B, for example, a combination of an icon / button identification “IcnKy3” representing activation of the web browser application and a relative position “above” is associated with the activation code “MnKy3-1”. For example, a combination of an icon / button identification “IcnKy3” indicating activation of the web browser application and a relative position “lower” is associated with the activation code “MnKy3-2”. For example, the combination of the icon / button identification “IcnKy3” representing the activation of the web browser application and the relative position “right” is associated with the activation code “MnKy3-3”.

  For example, on the display screen of the touch panel display device 110, the user generates a first contact position TP1 by touching an area above an icon / button for activating a desired “browser”. Next, as described above, the user keeps the first contact position TP1 or cancels the first contact position TP1, and touches the icon button representing “browser” with his / her finger to place the second contact position TP2. Then, the finger contact is released to release the second contact position TP2. In accordance with the operation, processor 102 activates the corresponding web browser application and activates the corresponding activation item “page of YH” based on the tables of FIGS. 21B and 21A, ie, corresponding web page acquisition request. Is transmitted via the network 5. Accordingly, a display screen of “YH page” is displayed as the web page screen.

  Further, for example, on the display screen of the touch panel display device 110, the user generates a first contact position TP1 by bringing a finger into contact with an area to the right of an icon / button for activating a desired “browser”. Next, as described above, the user keeps the first contact position TP1 or cancels the first contact position TP1, and touches the icon button representing “browser” with his / her finger to place the second contact position TP2. Then, the finger contact is released to release the second contact position TP2. In accordance with that operation, processor 102 activates the corresponding web browser application and activates the corresponding activation item “blog page” based on the tables of FIGS. 21B and 21A, ie, the corresponding web page retrieval request. Is transmitted via the network 5. As a result, a “blog page” display screen is displayed as the web page screen.

  As described above, in the touch panel display device 110, it is possible to reduce an operation procedure and an operation time for activating a desired activation item such as a process of acquiring and displaying a desired web page of a web browser. .

  FIG. 22A shows another example of a table of activation items for each activation code. FIG. 22B shows another example of the activation code table associated with the combination of the icon button at the second contact position TP2 and the relative position of the first contact position TP1 with respect to the second contact position TP2. Yes.

  In FIG. 22A, for example, the activation code MnKy3-1 is associated with an activation item “animation” in a moving image site (for example, YouTube) on the web. For example, the activation code MnKy3-2 is associated with the activation item “music” in a certain video site on the web. For example, the activation code MnKy3-3 is associated with the activation item “movie” in a certain video site on the web. In this case, the activation item may represent a search keyword in the video site search function.

  In FIG. 22B, for example, a combination of an icon / button identification “IcnKy3” indicating activation of a moving image site by a web browser application and a relative position “above” is associated with an activation code “MnKy3-1”. Yes. For example, a combination of an icon / button identification “IcnKy3” representing activation of a moving image site by a web browser application and a relative position “lower” is associated with the activation code “MnKy3-2”. For example, a combination of an icon / button identification “IcnKy3” representing activation of a moving image site by a web browser application and a relative position “right” is associated with the activation code “MnKy3-3”.

  For example, on the display screen of the touch panel display device 110, the user generates a first contact position TP1 by touching a region above an icon / button for activating a desired “moving image site”. Next, as described above, the user keeps the first contact position TP1 or cancels the first contact position TP1, and touches the icon button representing the desired “moving image site” with the finger to make the second The contact position TP2 is generated, and then the finger contact is released to release the second contact position TP2. In accordance with that operation, processor 102 activates the corresponding web browser application and activates the “animation” web page of the corresponding animation site based on the tables of FIGS. 22B and 22A. For this purpose, the processor 102 may transmit, for example, a web page acquisition request of the corresponding moving image site together with the search keyword “animation”. Thereby, in the information processing apparatus 10, a display screen of a list of “animation” videos is acquired and displayed as a web page screen.

  In this manner, in the touch panel display device 110, a desired activation item that activates and displays a web page of a search result using a desired search keyword for a desired moving image site using a web browser is activated. Therefore, the operation procedure and operation time can be reduced.

  FIG. 23A shows another example of a table of activation items for each activation code. FIG. 23B shows another example of the table of activation codes associated with the combination of the icon button at the second contact position TP2 and the relative position of the first contact position TP1 with respect to the second contact position TP2. Yes.

  In FIG. 23A, for example, the activation code MnKy3-1 is associated with the activation of the game application using the first saved data “save data 1” last time by user or by player. For example, the activation code MnKy3-2 is associated with the activation of the game application using the second saved data “save data 2” last time by user. For example, the activation code MnKy3-3 is associated with the activation of the game application using the third saved data “save data 3” last time by user.

  In FIG. 23B, for example, a combination of an icon / button identification “IcnKy3” representing activation of a game application and a relative position “above” is associated with an activation code “MnKy3-1”. For example, the combination of the icon / button identification “IcnKy3” representing the activation of the game application and the relative position “lower” is associated with the activation code “MnKy3-2”. For example, a combination of an icon / button identification “IcnKy3” representing activation of the game application and a relative position “right” is associated with the activation code “MnKy3-3”.

  For example, on the display screen of the touch panel display device 110, the user touches the area above the icon button for activating the game application with the desired “save data 1” to place the first contact position. TP1 is generated. Next, as described above, the user holds the first contact position TP1 or cancels the first contact position TP1, and touches the icon button representing a desired “game” with the finger to make the second contact. A position TP2 is generated, and then the finger contact is released to release the second contact position TP2. In accordance with that operation, processor 102 activates the corresponding game application with the desired “save data 1” based on the tables of FIGS. 23B and 23A to display the corresponding screen.

  As described above, in the touch panel display device 110, it is possible to reduce an operation procedure and an operation time for activating a desired activation item that activates or resumes a desired game application with personalized save data. it can.

  24A to 24C show another example of an operation procedure for activating a desired activation item in the information processing apparatus 10 in the single touch operation mode.

  Referring to FIG. 24A, the user activates the moving image folder of the file management tool by touching and releasing the finger on the “moving image folder” icon button 205 on the display screen of the touch panel display device 110. As a result, as shown in FIG. 24B, a list of various moving image folders is displayed as the activation item on the file management tool screen. In the display screen of FIG. 24B, the user activates the “Movie 9-Animation” folder of the file management tool by touching and releasing the finger on the button of the activation item “Movie 9-Animation” folder. As a result, as shown in FIG. 24C, a list of files of the activation items “animation 91-animation” to “animation 99-animation” in the folder of the file management tool is displayed. The user selects and activates the desired activation item file from the list of FIG. 24C by touching and releasing the finger on the button. Thereby, in the information processing apparatus 10, the corresponding operation reproduction application is activated, and the moving image of the file is reproduced.

  25A and 25B show another example of an operation procedure for activating a desired activation item in the information processing apparatus 10 in the multiple touch operation mode.

  FIG. 26A shows an example of an activation item table for each activation code as applied to FIGS. 25A and 25B. FIG. 26B shows an activation code associated with the combination of the icon button at the second contact position TP2 and the relative position of the first contact position TP1 with respect to the second contact position TP2 applied to FIGS. 25A and 25B. An example of the table is shown.

  In FIG. 26A, for example, the activation code MnKy3-1 is associated with the activation item “animation” of the file management tool. In this case, the activation item represents a moving image folder in the file management tool. For example, the activation code MnKy3-2 is associated with the activation item “music” of the file management tool. For example, the activation code MnKy3-3 is associated with the activation item “music” of the file management tool. In this case, the activation item may represent a search keyword used for the search function of the file management tool.

  In FIG. 26B, for example, a combination of an icon / button identification “IcnKy3” representing activation of a moving image folder of the file management tool and a relative position “above” is associated with the activation code “MnKy3-1”. For example, the combination of the icon / button identification “IcnKy3” representing the activation of the moving image folder of the file management tool and the relative position “lower” is associated with the activation code “MnKy3-2”. For example, a combination of an icon / button identification “IcnKy3” representing activation of a moving image folder of the file management tool and a relative position “right” is associated with the activation code “MnKy3-3”.

  Referring to FIG. 25A, on the display screen of the touch panel display device 110, the user is in a position that is in a certain relative positional relationship with the icon button 205 for activating the desired “moving image folder” of the file management tool. The first contact position TP1 is generated by touching the finger. Next, as described above, the user keeps the first contact position TP1 or cancels the first contact position TP1, and touches the icon button 205 of the “moving image folder” with the finger to move to the second contact position. TP2 is generated, and then the finger contact is released to release the second contact position TP2. In accordance with that operation, the processor 102, based on the tables of FIGS. 26B and 26A, activates the subordinate activation items “Animation 91-Animation”-“in the animation folder of the file management tool as shown in FIG. 25B. A list of files of “Movie 99-Anime” is displayed. FIG. 25B corresponds to FIG. 24C. That is, the intermediate activation item or moving image folder display screen of FIG. 24B is skipped, and the desired activation item or folder file display screen of FIG. 25B or 24C is displayed.

  As described above, in the touch panel display device 110, the operation procedure and operation time for activating a desired activation item for searching and displaying a desired folder or file in the file management tool with a desired search keyword is reduced. Can be made.

In FIG. 1, the processor 102 or its contact detection unit 1122 (FIG. 2) determines the coordinates (x, y) of the position of one or more contacts (touch) and the contact state or its state change from the touch panel display device 110. Receive data to represent and detect changes. When the state change is detected, a position information event is generated in the contact detection unit 1122. Each contact state or its state change SC includes the following four.
(1) The occurrence of a new contact (1), ie a change SC (0-1) from the absence (0) of some contact to the contact state (1),
(2) Maintenance of the previous contact (1), that is, the contact state (1) at a previous position of a certain contact is maintained as the current contact state (1) at the same position SC (1-1) ,
(3) Movement of contact (1), that is, a change SC (1-2) from a contact state (1) at a previous position to a contact state (2) at another position, and (4) contact The disappearance of (1), that is, the change SC (1-0) from the contact state (1) of a certain contact to the non-contact state (0).

  As an alternative, the data of the state change SC may be generated by the contact detection unit 1122 based on data representing the coordinates (x, y) and the contact state (0 or 1) from the touch panel display device 110.

  The contact detection unit 1122 determines or classifies the contact state of each contact or data indicating the state change based on the data representing the state change SC, and coordinates (x, y) of each contact position and the contact. Data representing states 1 to 6 is generated. The contact detection unit 1122 may pass the generated data to the first contact state determination unit 1124 and further to the second contact state determination unit 1126. It can be said that the contact states 1 to 6 represent one or two contact state transitions.

  In the time difference multiple touch operation mode, the contact detection unit 1122 selects either the timer-on mode using the timer 1128 or the timer-off mode not using the timer 1128 according to the presence / absence of the timer-off mode stored in the timer mode storage area. Can work with. As will be described later, the timer mode setting unit 1130 stores the timer off mode in the timer mode storage area such as the memory 104 or the timer mode storage area according to the user's setting operation (FIGS. 6A and 10A). The timer off mode in may be deleted.

  FIG. 27 is a flowchart for determining contact states 1 to 6 of one or two contacts (touch) executed by the processor 102 of the information processing apparatus 10 or its contact detection unit 1122 in the simultaneous multiple touch operation mode. An example is shown.

  In step 402, the processor 102 or the contact detection unit 1122 waits for occurrence of a position information event from the touch panel display device 110. When the position information event occurs, the contact detection unit 1122 receives the coordinates (x1, y1) of the first contact position TP1 and / or the coordinates (x2, y2) of the second contact position TP2 from the touch panel display device 110 and the respective states. And data representing the SC.

  In step 404, the contact detection unit 1122 processes the received data and determines whether the state of the first contact position TP1 and / or the second contact position TP2 is the contact state 1-6.

  If the new first contact position TP1 is generated and the state change SC (0-1) is generated without the second contact position, the contact detection unit 1122 touches the state of the touch operation in Step 412. State 1 is determined. When the first contact position TP1 moves and the state change SC (1-2) is generated without the second contact position, the contact detection unit 1122 changes the touch operation state to the contact state in Step 414. 2 is determined. When the first contact position TP1 disappears and the state change SC (1-0) is generated without the second contact position, the contact detection unit 1122 changes the touch operation state to the contact state in Step 416. 3 is determined.

  On the other hand, while maintaining the first contact position TP1 (SC (1-1)), the second contact position TP2 is generated (SC (0-1)), and the state change SC (1-1, 0-1) is generated. If it is determined, the contact detection unit 1122 determines that the touch operation state is the contact state 4 in step 418. Here, the state change SC (1-1, 0-1) represents a combination of state changes of the first contact position TP1 and the second contact position TP2. One of the first contact position TP1 and the second contact position TP2 maintains the contact position (SC (1-1)) and the other moves (SC (1-2)), and the state change SC (1-1, 1) -2) is generated, the contact detection unit 1122 determines that the touch operation state is the contact state 5 in step 420. One of the first contact position TP1 and the second contact position TP2 remains in contact (SC (1-1)) and the other disappears (SC (1-0)), and the state change SC (1-1, 1- 0) is generated, the contact detection unit 1122 determines that the touch operation state is the contact state 6 in step 422.

  In Step 432 after Steps 412 to 422, the contact detection unit 1122 passes the first contact position TP <b> 1 and the second contact position TP <b> 2 and information indicating the contact states 1 to 6 to the first contact state determination unit 1124. Further, it may be passed to the second contact state determination unit 1126. Alternatively, the processor 102 determines the contact state between the first contact position TP1 and the second contact position TP2 of the processor 102 based on the first contact position TP1 and the second contact position TP2 and information indicating the contact states 1 to 6 thereof. May be passed to each process.

  28A to 28D show examples of flowcharts for determining each process according to the contact state of the first contact position TP1, which is executed by the processor 102 or the first contact state determination unit 1124 thereof.

  In step 502, the first contact state determination unit 1124 determines the contact state between the first contact position TP <b> 1 and the second contact position TP <b> 2 based on information representing any one of the contact states 1 to 6 generated by the contact detection unit 1122. Determine if has changed.

  When information representing any one of the contact states 1 to 6 is generated, in step 504, the first contact state determination unit 1124 determines whether the state is any of the contact states 1 to 3. When it is determined that the state is any one of the contact states 1 to 3, that is, when the state change is related to only the first contact position TP1, the procedure proceeds to step 512 in FIG. 28B. When it is determined that the state is not any of the contact states 1 to 3, that is, when the state change is related to the combination of the first contact position TP <b> 1 and the second contact position TP <b> 2, in Step 506, the first contact state determination unit 1124 It is determined whether or not the first contact position TP1 is the same as the previous position. If it is determined that the positions are not the same, the procedure proceeds to S3 in FIG. 28C. If it is determined that the position is the same and the second contact position has changed, the procedure proceeds to step 508. In step 508, the first contact state determination unit 1124 receives information representing a change in the contact state, that is, information representing the first contact position TP1, the second contact position TP2, and the state SC or the contact states 4 to 6 as the second. It passes to the contact state determination part 1126. Alternatively, the processor 102 may pass such information to a process for determining the contact state of the second contact position TP2.

  Referring to FIG. 28B, in step 512, the first contact state determination unit 1124 determines which of the contact states 1 to 3 the state is.

  When it is determined that the state is the contact state 1, in step 522, the first contact state determination unit 1124 stores data representing the first contact position TP1 and the contact state 1 in, for example, the memory 104, the storage device 130, and the like. Store in the first contact position storage area.

  In step 526, the first contact state determination unit 1124 determines whether or not the first contact position TP1 is on any icon button compared with the coordinate area of each icon button. If it is determined that the position is on the icon button, the procedure proceeds to step 528. In step 528, the first contact state determination unit 1124 may highlight or pick up the icon / button of the first contact position TP1. If it is determined that the position is not on any of the icons and buttons, in step 530, the first contact state determination unit 1124 picks up and displays the generated first contact position TP1 with, for example, a circular symbol (208). May be. Thereafter, the procedure returns to step 502 in FIG. 28A.

  When it is determined in step 512 that the state is the contact state 2, in step 542, the first contact state determination unit 1124 determines whether or not the first contact position TP1 is on the same icon / button as the previous time. . If it is determined that the position is on the same icon / button as the previous time, the procedure proceeds to step 556. When the first contact position TP1 is on the icon button, the highlighted display is maintained even if the position moves on the same icon button.

  If it is determined in step 542 that the position is not on the same icon or button as the previous time, in step 544, the first contact state determination unit 1124 determines that the previous first contact position TP1 is on any icon or button. It is determined whether it was located at. If it is determined that the previous first contact position TP1 was located on the icon / button, in step 546, the first contact state determination unit 1124 cancels the highlight display of the previous icon / button. If it is determined that the previous first contact position TP1 is not located on the icon button, in step 548, the first contact state determination unit 1124 displays a pickup display of the previous first contact position TP1, that is, a circular symbol. The display (208) is canceled.

  Steps 526 to 530 are the same as in the contact state 1. Thereby, a new icon / button of the first contact position TP1 after the movement may be highlighted, or the first contact position TP1 after the movement may be picked up and displayed. Thereafter, the procedure proceeds to step 556.

  In step 556, the first contact state determination unit 1124 stores the data indicating the first contact position TP1 after movement and the contact state 2 in the first contact position storage area such as the memory 104 or the storage device 130, for example. Update. Thereafter, the procedure returns to step 502 in FIG. 28A.

  When it is determined in step 512 that the state is the contact state 3, in step 562, the first contact state determination unit 1124 determines that the first contact position TP1 disappeared from the first contact position storage area such as the memory 104 and the like. The data representing the contact state is deleted and updated. Steps 544 to 548 are the same as in the contact state 2. Thereby, the highlighted display of the icon / button of the disappeared first contact position TP1 is canceled, or the pickup display of the disappeared first contact position TP1 is canceled or erased. After step 548, the procedure returns to step 502 of FIG. 28A.

  When it is determined in step 544 in the case of the contact state 3 that the previous first contact position TP1 is located on the icon / button, the disappearance of the first contact position TP1 is performed by clicking the icon / button with a single touch operation. It represents that. In Step 568 after Step 546, the first contact state determination unit 1124 notifies the activation processing unit 1146 or the application unit 1022 of the previous icon / button press event at the first contact position TP 1. The activation processing unit 1146 may request the application unit 1022 to execute the activation item represented by the icon button. Thereafter, the procedure returns to step 502 in FIG. 28A.

  Referring to FIG. 28C, in step 582 after the NO (negative) branch of step 506 in FIG. 28A, the first contact state determination unit 1124 determines whether the state is the contact state 5 or 6. .

  If it is determined that the state is the contact state 5, the procedure proceeds to Step 542. Steps 542 to 548, 526 to 530, and 556 are the same as those in the contact state 2 of FIG. 28B.

  After step 556 after step 528 or 530, in step 760, the first contact state determination unit 1124 includes a relative position determination unit 1134, an activation item selection unit 1142, an activation item display unit 1144, and an auxiliary line display unit 1132. In addition, the presence or state change of the first contact position TP1 and the second contact position TP2 is notified (see FIG. 30). Thereafter, the procedure exits the routine of FIG. 28C and returns to step 502 of FIG. 28A.

  In FIG. 28C, when it is determined in step 542 that the previous first contact position TP1 is located on another icon / button, the procedure exits the routine of FIG. 28C and returns to step 502 of FIG. 28A. Alternatively, before exiting the routine of FIG. 28C, step 556 may be performed as indicated by the dashed line. In this case, data representing the contact state 5 with the first contact position TP1 is stored in the first contact position storage area such as the memory 104 and updated. However, step 556 need not be executed, and as long as the first contact position TP1 is located on the same icon / button as before, the first contact position TP1 in the first contact position storage area such as the memory 104 is not updated. May be. Thereby, the processing load is reduced.

  Referring to FIG. 28D, if it is determined in step 582 of FIG. 28C that the state is contact state 6, the procedure proceeds to step 544. Steps 544 to 548 are the same as those in the contact state 2 of FIG. 28B. Thereby, the highlighted display of the icon / button of the disappeared first contact position TP1 is canceled, or the pickup display of the disappeared first contact position TP1 is canceled or erased.

  In step 608, the first contact state determination unit 1124 deletes the previous first contact position TP1 that disappeared and data representing the contact state from the first contact position storage area.

  In step 610, the first contact state determination unit 1124 uses the data representing the second contact position TP2 and the contact state 6 in the second contact position storage area such as the memory 104 as the first first contact position TP1 as the first contact position. Store in the storage area and update. Thereafter, the procedure proceeds to step 526.

  Step 526 is the same as in the contact state 1 of FIG. 28B. When the previous second contact position TP2 before the update is positioned on the icon button, the updated first contact position TP1 corresponding to the second contact position TP2 is already highlighted. Therefore, after step 526, the highlight display as in step 528 in FIG. 28B may not be executed. If it is determined in step 526 that the updated first contact position TP1 is on the icon button, in step 616, the first contact state determination unit 1124 displays the activation item display unit 1144 on the disappearance of the second contact position TP2. To be notified. In response to the notification, the activation item display unit 1144 cancels the activation item display 207 of the activation item (FIGS. 8B, 8D, 9A, 9B, 10C, and 10D). In step 618, the first contact state determination unit 1124 notifies the auxiliary line display unit 1132 of the cancellation of the auxiliary line display at the second contact position TP2. In response to the notification, the auxiliary line display unit 1132 cancels the auxiliary line display (FIGS. 11A to 11D and 220 to 226). Thereafter, the procedure proceeds to Step 624.

  If it is determined in step 526 that the updated first contact position TP1 is not on the icon button, step 530 similar to FIG. 28B is executed. Here, when the second contact position TP2 before update is not picked up and displayed, the first contact state determination unit 1124 picks up the updated first contact position TP1 corresponding to the second contact position TP2 before update. It may be displayed. Thereafter, the procedure proceeds to Step 624.

  In step 624, the first contact state determination unit 1124 deletes and updates the data representing the second contact position TP2 and the contact state from the second contact position storage area such as the memory 104. Thereafter, the procedure exits the routine of FIG. 28D and returns to step 502 of FIG. 28A.

  29A to 29C show examples of flowcharts for determining each process according to the contact state of the second contact position TP2, which is executed by the processor 102 or the second contact state determination unit 1126 thereof.

  In step 632, in response to the notification from the first contact state determination unit 1124 (FIG. 28A, step 508), the second contact state determination unit 1126 represents the contact states 4 to 6 generated by the contact detection unit 1122. It is determined whether the contact state of the second contact has changed based on the information.

  When information representing any one of the contact states 4 to 6 is generated, in step 634, the second contact state determination unit 1126 determines whether the state is any of the contact states 4 to 6. If it is determined that the state is the contact state 4, the procedure proceeds to Step 636. If it is determined that the state is the contact state 5, the procedure proceeds to Step 652 in FIG. 29B. If it is determined that the state is the contact state 6, the procedure proceeds to Step 702 in FIG. 29C.

  In step 636, the second contact state determination unit 1126 stores the generated second contact position TP2 and data representing the contact state 4 in a second contact position storage area such as the memory 104 or the storage device 130, for example.

  In step 640, the first contact state determination unit 1124 determines whether the position of the second contact position TP <b> 2 is on any icon button compared with the coordinate area of each icon button. If it is determined that the position is not on the icon button, the procedure returns to step 632. If it is determined that the position is on the icon button, the procedure proceeds to step 760 (see FIG. 30). Step 760 is similar to that of FIG. 28C. Thereafter, the procedure returns to Step 632.

  Referring to FIG. 29B, if it is determined in step 634 in FIG. 29A that the state is the contact state 5, in step 652, the second contact state determination unit 1126 determines that the current second contact position TP2 is the same as the previous time. Determine if it is over an icon button. If it is determined that the position is on the same icon / button as the previous time, the procedure proceeds to step 680.

  If it is determined in step 652 that the position is not on the same icon / button as the previous time, in step 654, the second contact state determination unit 1126 determines that the previous second contact position TP2 is positioned on the icon / button. Determine whether you were doing. If it is determined that the previous second contact position TP2 is located on the icon button, in step 656, the second contact state determination unit 1126 determines that the second contact position TP2 is an activation item (FIGS. 8B and 8D). 5A, 5B, 6C, 6D) is determined. If it is determined that the position is on the activation item display 207, the procedure proceeds to step 680.

  If it is determined in step 656 that the position is not on the activation item display 207, in step 658, the second contact state determination unit 1126 displays the activation item of the second contact position TP2 on the activation item display unit 1144. Notify the cancellation of the display. In response to the notification, the activation item display unit 1144 cancels the activation item display 207. In Step 660, the second contact state determination unit 1126 notifies the auxiliary line display unit 1132 of the cancellation of the auxiliary line display at the second contact position TP2. The auxiliary line display unit 1132 cancels the display of auxiliary lines (FIGS. 11A to 11D and 220 to 226). Thereafter, the procedure proceeds to Step 680.

  If it is determined in step 654 that the previous second contact position TP2 was not positioned on the icon button, in step 670, the second contact state determination unit 1126 determines that the current second contact position TP2 is an icon. -Determine if it is on the button. If it is determined that the position is not on the icon button, the procedure proceeds to step 680. If it is determined that the position is on another icon button, the procedure proceeds to step 760 (see FIG. 30). Step 760 is similar to that of FIG. 28C. Thereafter, the procedure proceeds to Step 680.

  In step 680, the second contact state determination unit 1126 stores and updates the second contact position TP2 after movement and the data representing the contact state 6 in the second contact position storage area such as the memory 104. Thereafter, the procedure returns to step 632 of FIG. 29A.

  Referring to FIG. 29C, if it is determined in step 634 of FIG. 29A that the state is the contact state 6, in step 702, the second contact state determination unit 1126 determines that the previous second contact position TP2 is an icon button. Determine if it was located above. If it is determined that the previous second contact position TP2 is located on the icon / button and has disappeared this time and a click has occurred, in step 704, the second contact state determination unit 1126 determines the occurrence of the operation event as a relative position. The determination unit 1134 and the activation item selection unit 1142 are notified.

  In response to the notification in step 704, the relative position determination unit 1134 determines the relative position of the first contact position TP1 with respect to the previous second contact position TP2 released on the icon button, and the activation item selection unit 1142 is passed. The activation item selection unit 1142 is based on the activation code table corresponding to the combination of the icon button and the relative position of the first contact position TP1, and the activation item table corresponding to the activation code (see FIG. 3A, 3B), the corresponding activation item of the corresponding application is selected. The activation item selection unit 1142 passes identification information representing the corresponding application and the selected activation item to the activation processing unit 1146. The activation processing unit 1146 activates the corresponding activation item in the corresponding application in the application unit 1022.

  When the activation item corresponding to the relative position is acquired in advance in step 774 of FIG. 30 described later, in step 704, the second contact state determination unit 1126 determines the occurrence of the operation event as the activation item selection unit 1142. Alternatively, the activation processing unit 1146 may be notified. In response to the notification, the activation item selection unit 1142 passes identification information representing the corresponding application and the activation item acquired in Step 774 of FIG. 30 to the activation processing unit 1146.

  As an alternative, if the second touch position TP2 is not processed as an operation event unless it is released on the activation item display (207) around the icon button, step 704 is not executed and the procedure is YES in step 702. (Yes) After branching, the process may proceed to step 706.

  In step 706, the second contact state determination unit 1126 notifies the activation item display unit 1144 that the second contact position TP2 has disappeared. In response to the notification, the activation item display unit 1144 cancels the activation item display (207). In step 708, the second contact state determination unit 1126 notifies the auxiliary line display unit 1132 that the second contact position TP2 has disappeared. In response to the notification, the auxiliary line display unit 1132 cancels the display of the auxiliary line. Thereafter, the procedure proceeds to Step 720.

  If it is determined in step 702 that the previous second contact position TP2 is not located on the icon button, in step 712, the second contact state determination unit 1126 determines that the previous second contact position TP2 is an icon button. It is determined whether it is located on the activation item display (207) around the. If it is determined that the position is located on the activation item display 207, the procedure proceeds to step 714. If it is determined that the position is not located on the activation item display (207), the procedure proceeds to step 720.

  In step 714, the second contact state determination unit 1126 notifies the activation processing unit 1146 that an operation event has occurred. The occurrence of the operation event may be notified together with the identification information of the activation item in the activation item display (207) of the released second contact position TP2.

  In response to the notification in step 714, the activation processing unit 1146 selects the notified activation item or one activation item selected by the activation item selection unit 1142 in the processing of FIG. 30 for step 760. The application unit 1022 is requested to execute. The application unit 1022 activates the corresponding activation item in the corresponding application. As an alternative, if the second touch position TP2 is not processed as an operation event when it is released on the activation item display (207) around the icon button, steps 712 and 714 are not executed and the procedure Proceed to 720.

  In step 720, the second contact state determination unit 1126 deletes the second contact position TP2 and data representing the contact state from the second contact position storage area such as the memory 104.

  FIG. 30 shows the processor 102 or its relative position determination unit 1134, the activation item selection unit 1142, the activation item display unit 1144, and the auxiliary line display unit 1132 in response to the notification in step 760 of FIGS. 28C, 29A, and 29B. The example of the flowchart of the process performed is shown.

  In step 772, the relative position determination unit 1134 determines the first contact position from the coordinates (x1, y1) and (x2, y2) of the confirmed first contact position TP1 and second contact position TP2 notified in step 760. A relative position of TP1 with respect to the second contact position TP2 is obtained. The relative position is, for example, an azimuth range and / or a distance range as shown in FIGS.

  In step 774, the activation item selector 1142 determines from the determined relative position one corresponding activation associated with the icon button of the second contact position TP2, for example based on the table shown in FIG. 3B. Get the code. Next, the activation item selection unit 1142 determines identification information of one or a plurality of activation items corresponding to the obtained activation code based on, for example, the tables shown in FIGS. 3A and 7A, for example, It may be temporarily stored in a storage area such as the memory 104 or the storage device 130.

  In step 776, the activation item display unit 1144 displays the retrieved activation item or items around the icon button. When there is only one extracted activation item, the activation item display unit 1144 may not display the activation item. Each activation item may be displayed adjacent to the icon button in an arrangement corresponding to its relative position around it. If no activation item is displayed, step 776 is not performed and steps 712 and 714 are not performed in FIG. 29C, but step 702 is performed.

  In step 778, the auxiliary line display unit 1132 may display the auxiliary lines 220 to 226 such as radial or concentric circles shown in FIGS. 11A to 11D with the second contact position TP2 as the center or the origin. However, if the auxiliary lines 220 to 226 are not displayed, step 778 is not executed.

  FIG. 31A shows an example of a flowchart for processing executed by the processor 102 or its relative position determination unit 1134, the activation item selection unit 1142, and the activation processing unit 1146 in response to step 704 of FIG. 29C. ing.

  In step 780, the relative position determination unit 1134 determines the relative position of the first contact position TP1 with respect to the previous second contact position TP2 released on the icon / button. However, if step 772 of FIG. 30 is executed in advance, step 780 may not be executed.

  In step 782, the activation item selection unit 1142 determines the combination of the released icon / button identification of the second contact position TP2 and the relative position of the first contact position TP1, for example, based on the table shown in FIG. 3B. , Obtain one corresponding activation code. Next, the activation item selection unit 1142 determines identification information of one activation item corresponding to the acquired activation code, for example, based on the table shown in FIG. 3A. However, when step 774 of FIG. 30 is executed in advance, step 782 may not be executed.

  In step 784, the activation item selection unit 1142 requests the application unit 1022 to activate the one activation item determined in step 782 or step 774 of FIG. The application unit 1022 activates the corresponding activation item of the corresponding application.

  FIG. 31B shows an example of a flowchart for processing executed by the processor 102 or its activation processing unit 1146 corresponding to step 714 of FIG. 29C.

  In step 783, the activation processing unit 1146 uses the notified identification information of one activation item or one of the items determined based on the relative position by the activation item selection unit 1142 in the processing of FIG. 30 for step 760. Get identification information of activation item. Step 784 is similar to that of FIG. 31A. Thereby, the corresponding activation item of the corresponding application is activated by the application unit 1022.

  In this manner, in the simple simultaneous two-touch operation mode, one or a plurality of activation items are determined and selected from the icon / button of the second contact position and the relative position of the first contact position TP1. One activated item can be activated.

  32A to 32C show examples of screens for registering the correspondence between the combination of the relative positions of the icons and buttons and the first contact position TP1 and the activation items. As a result of the registration, as shown in FIGS. 3B and 7B, one of the applications corresponding to the combination of the identification of the icon / button selected at the second contact position TP2 and the relative position of the first contact position TP1. Activation codes corresponding to one or more activation items are created.

  The user clicks and selects an icon button (not shown) for registering the correspondence on the display screen of the touch panel display device 110. As a result, the correspondence setting unit 1160 of the processor 102 is activated, and a screen as shown in FIG. 32A is displayed on the touch panel display device 110, for example.

  In FIG. 32A, the user clicks and selects an icon button (for example, mailer) of an application to be activated by a two-touch operation. Then, a list of activation items that can be associated with icon buttons is displayed as shown in FIG. 32B. The user clicks and selects one or more activation items in the list that he or she wants to activate with a two touch operation for the selected icon button.

  Then, as shown in FIG. 32C, the selected activation item or items and relative positions (for example, “up / down / left / right”, “distance range 1/2/3” that can be associated with each of the activation items. / 4 ") is displayed. The user clicks and selects a relative position to be associated with each activation item and registers it. If one or more activation items are selected at one icon button and one relative position, the activation items are displayed so that selectable activation items are displayed, eg, FIGS. 7A, 7B. May be registered in the form of a table. When one activation item is selected at one icon button and one relative position, the activation item may be registered in the form of a table such as FIG. 3A, 3B, 14A, 14B, etc. .

  33A and 33B determine contact states 1 to 6 of one or two contacts (touch) executed by the processor 102 of the information processing apparatus 10 or its contact detection unit 1122 in the time difference multiple touch operation mode using a timer. The example of the flowchart for doing is shown.

  In step 802, the processor 102 or the contact detection unit 1122 waits for occurrence of a position information event from the touch panel display device 110 and a timer event from the timer 1128. When the position information event occurs, the contact detection unit 1122 receives the coordinates (x, y) of the contact position and data indicating the state SC from the touch panel display device 110. The procedure then proceeds to step 804. On the other hand, if a timer event has occurred, the procedure proceeds to step 860 in FIG. 33B.

  When a position information event occurs, in step 804, the contact detection unit 1122 stores the determined first contact position TP1 and its contact state 3 in the first contact position storage area for timer setting such as the memory 104. Determine whether it has been. When it is determined that the confirmed first contact position TP1 and its contact state 3 are not stored, the contact detection unit 1122 uses the coordinates (x, y) of the received contact position as the coordinates of the first contact position TP1 ( x1, y1). Thereafter, the procedure proceeds to Step 806.

  When it is determined that the first contact position TP1 and the contact state 3 determined in step 804 are stored, the contact detection unit 1122 uses the coordinates (x, y) of the received contact position of the second contact position TP2. It is determined that the coordinates are (x2, y2). Thereafter, the procedure proceeds to Step 808.

  In step 806, the contact detection unit 1122 determines which of the contact states 1 to 3 is the first contact position TP1. As described above, when the first contact position TP1 is generated and the state change SC (0-1) is generated, the contact detection unit 1122 determines that the touch operation state is the contact state 1 in step 412. To do. When the first contact position TP1 moves and the state change SC (1-2) is generated, the contact detection unit 1122 determines that the touch operation state is the contact state 2 in step 414. When the first contact position TP1 disappears and the state change SC (1-0) is generated, the contact detection unit 1122 determines that the touch operation state is the contact state 3 in step 416.

  In Step 832 after Steps 812 and 814, the contact detection unit 1122 sends the first contact position TP1 and information indicating one of the determined contact states 1 and 2 to the first contact state determination unit 1124. hand over. Alternatively, the processor 102 uses the first contact position TP1 and the information indicating one of the determined contact states 1 and 2 to each process for determining the contact state of the first contact position TP1 of the processor 102. You may pass. Thereafter, the procedure returns to Step 802.

  In Step 836 after Step 816, the contact detection unit 1122 stores information representing the first contact position TP1 and the contact state 3 in the first contact position storage area for state determination such as the memory 104. The first contact position TP1 in the contact state 3 represents a determined position. In step 840, the timer 1128 is set to measure a time limit (eg, 0.5 seconds or 1 second) for detecting the second contact. Thereafter, the procedure returns to Step 802.

  In step 808, since a change has occurred with respect to the second contact position before the time limit for detecting the second contact in the timer 1128 has elapsed, the contact detection unit 1122 cancels the setting of the timer 1128 for measuring the time. Or stop its operation.

  In step 810, the contact detection unit 1122 determines which of the contact states 4 to 6 is the second contact position TP2.

  When the second contact position TP2 occurs and the state change SC (0-1) is generated, the contact detection unit 1122 determines that the touch operation state is the contact state 4 in step 822. When the second contact is moved and the state change SC (1-2) is generated, the contact detection unit 1122 determines that the touch operation state is the contact state 5 in Step 824. When the second contact disappears and the state change SC (1-0) is generated, the contact detection unit 1122 determines that the touch operation state is the contact state 6 in Step 826. The second contact position TP2 in the contact state 6 represents a confirmed position.

  In Step 852 after Steps 822 and 824, the contact detection unit 1122 stores information representing the second contact position TP2 and the contact state 4 or 5 in the second contact position storage area for state determination such as the memory 104. Update.

  In Step 854 after Step 826, the contact detection unit 1122 deletes the second contact position TP2 in the second contact position storage area for state determination such as the memory 104 and information indicating the contact state.

  In Step 858 after Steps 852 and 854, the contact detection unit 1122 uses the second contact position determination unit 1126 and the second contact state determination unit 1126 to obtain the second contact position TP2 and information indicating any of the determined contact states 4 to 6. 1 contact state determination unit 1124. Alternatively, the processor 102 determines the contact state between the second contact position TP2 and the first contact position TP1 of the processor 102 based on the second contact position TP2 and information indicating any of the determined contact states 4 to 6. May be passed to each process. Thereafter, the procedure returns to Step 802.

  When a timer event indicating the elapse of the set time occurs in step 802, in step 860 of FIG. 33B, the contact detection unit 1122 stores the first stored in the first contact position storage area for state determination such as the memory 104. The coordinates (x1, y1) of the contact position TP1 and the contact state (1, 2) are read out.

  In step 862, the contact detection unit 1122 sends information indicating that the set time limit has elapsed for the first contact position TP1 in the contact state 1 or 2 to the first contact state determination unit 1124 or the first contact position of the processor 102. Notify the process for determining the contact state of TP1.

  In step 864, the first contact state determination unit 112 deletes the coordinates and the contact state of the confirmed first contact position TP1 being stored in the first contact position storage area for state determination such as the memory 104. Thereafter, the procedure exits the routine of FIG. 33B.

  FIG. 34 shows an example of a flowchart for determining processing according to the contact state of the first contact position TP1, which is executed by the processor 102 or the first contact state determination unit 1124 thereof.

  In step 502, in response to the notification from the contact detection unit 1122 (steps 832 and 862 in FIGS. 33A and 33B), the first contact state determination unit 1124 is based on the contact states 1 to 3 from the contact detection unit 1122. It is determined whether or not the contact state of the first contact position TP1 has changed.

  When information representing any one of the contact states 1 to 3 is received, in step 512, the first contact state determination unit 1124 determines whether the state is the contact state 1 to 3. If it is determined that the state is the contact state 1, the procedure proceeds to step 522. Steps 522 to 530 in this case are the same as those in the case of the contact state 1 in FIG. 28B. After steps 528 and 530, the procedure returns to step 502.

  If it is determined that the state is the contact state 2, the procedure proceeds to Step 542. Steps 542 to 548, 526 to 530, and 556 in this case are the same as those in the contact state 2 of FIG. 28B. After step 556, the procedure returns to step 502.

  If it is determined that the state is the contact state 3, the procedure proceeds to Step 562. In step 562, the first contact state determination unit 1124 stores the first contact position for determining the state in the memory 104 or the like because the timer setting time limit has elapsed after the contact state of the first contact position TP1 has been determined. Data representing the first contact position TP1 and the contact state 3 is deleted from the area and updated. Thereafter, the procedure proceeds to Step 544. Steps 544 to 548 and 568 in this case are the same as those in the case of the contact state 3 in FIG. 28B. After steps 568 and 548, the procedure returns to step 502.

  FIGS. 35A to 35C show an example of a flowchart for determining processing according to the contact state of the second contact position TP2, which is executed by the processor 102 or the second contact state determination unit 1126 thereof.

  In FIG. 35A, in response to the notification from the contact detection unit 1122 (FIG. 33A, step 858), in step 632, the second contact state determination unit 1126 changes the contact states 4 to 6 generated by the contact detection unit 1122 to each other. Based on this, it is determined whether the contact state of the second contact has changed. Steps 634 to 640 and 760 are the same as those in FIG. 29A. In FIG. 35B, steps 652 to 680 and 760 are the same as those in FIG. 29B. In FIG. 35C, steps 702 to 720 are the same as those in FIG. 29C.

  Referring to FIG. 35C, in step 872 after step 720, the second contact state determination unit 1126 determines whether or not the determined coordinates of the first contact position TP1 are on the icon button. If it is determined that the coordinates are on the icon / button, in step 874, the second contact state determination unit 1126 cancels highlight display of the icon / button. If it is determined that the coordinates are not on the icon / button, in step 876, the second contact state determination unit 1126 cancels the pickup display of the contact position. In Step 878 after Steps 874 and 876, the second contact state determination unit 1126 deletes and updates the first contact position TP1 and the contact state from the first contact position storage area such as the memory 104. As an alternative, steps 872 to 878 may be executed by the first contact state determination unit 1124. Thereafter, the procedure returns to step 632 of FIG. 35A.

  In this manner, one or a plurality of activation items are determined from the icon / button of the second contact position and the relative position of the first contact position TP1 by a time difference two-touch operation using a simple timer, Alternatively, one selected activation item can be activated.

  FIG. 36 shows an example of a flowchart for switching the timer mode set by the user, which is executed by the processor 102 or its timer mode setting unit 1130 in the time difference multiple touch operation mode without using the timer.

  In step 890, the processor 102 or the timer mode setting unit 1130 waits for the occurrence of a timer mode setting event. For example, the user switches the timer mode between the timer-on state and the timer-off state by clicking the icon button 210 for switching the timer mode on the display screens of FIGS. 6A and 6B.

  If a timer mode setting event has occurred, in step 892, the timer mode setting unit 1130 determines whether or not the timer mode is a timer off mode. When it is determined that the timer mode is the timer off mode, in step 894, the timer mode setting unit 1130 stores setting information indicating the timer off mode in a timer mode storage area such as the memory 104. If it is determined that the timer mode is not the timer off mode, in step 896, the timer mode setting unit 1130 deletes the setting information indicating the timer off mode from the timer mode storage area such as the memory 104. In step 898, the timer mode setting unit 1130 deletes the data representing the first contact position TP1, the second contact position TP2, and the contact state thereof in the first and second contact position storage areas such as the memory 104. Thereafter, the procedure exits the routine of FIG.

  FIG. 37 illustrates one or two contact (touch) contact states 1 to 6 executed by the processor 102 of the information processing apparatus 10 or its contact detection unit 1122 in the time difference multiple touch operation mode without using a timer. The example of the flowchart for this is shown.

  In step 801, the processor 102 or the contact detection unit 1122 waits for the occurrence of a position information event from the touch panel display device 110. When the position information event occurs, the contact detection unit 1122 receives the coordinates (x, y) of the contact position and data representing the state SC from the touch panel display device 110.

  In step 803, the contact detection unit 1122 refers to a timer mode storage area such as the memory 104 and determines whether the timer off mode is set. If it is determined that the timer off mode is not set, the process returns to step 801. When the timer off mode is not set, the information processing apparatus 10 operates in a time difference multiple touch operation mode (FIGS. 33A and 33B) using a timer. If it is determined that the timer off mode is set, the procedure proceeds to step 804. Steps 804, 806, 810-826, 832, 852-858 are similar to those of FIG. 33A. In FIG. 37, the processes related to the timers in steps 808 and 840 of FIG. 33A and steps 860 to 864 of FIG. 33B are not executed.

  The flowcharts of FIGS. 34 and 35A to 35C can also be applied to the time difference multiple touch operation mode that does not use the timer in the information processing apparatus 10 related to FIG. However, steps 562 to 568 in the contact state 3 in FIG. 34 are not executed. In this case, when it is desired to release the first contact position TP1 with respect to the contact state 3, for example, by switching the mode to the timer on mode in FIG. 36, the first contact position TP1 and the data representing the contact state are obtained in step 898. It can be deleted from the one-contact position storage area.

In this manner, one or a plurality of activation items are determined from the icon / button of the second contact position and the relative position of the first contact position TP1 by a time difference two-touch operation without using a simple timer, That or selected one activation item can be activated.
In the time difference multiple touch operation mode that does not use a timer, the user can perform a plurality of touch operations without worrying about the time between the two touch operations.

  38A and 38B show still another example of the operation procedure on the display screen of the touch panel display device 110 in the multiple touch operation mode according to the embodiment. The multiple touch operation mode may be any of the above-described simultaneous multiple touch operation mode, a time difference multiple touch operation mode using the timer 1128, and a time difference multiple touch operation mode not using a timer.

  Referring to FIG. 38A, the user generates a first touch position TP1 by touching a first position on the icon button 206 representing activation of the mailer application. Next, the user releases the first contact position TP1 in the time difference multiple touch operation mode while holding the first contact position TP1 in the simultaneous multiple touch operation mode. Next, referring to FIG. 38B, the user causes the second contact position TP2 to be generated by bringing the finger into contact with the second position on the left side of the first contact position on the icon button 206. Next, the user releases the finger at the second contact position TP2, and the second contact position TP2 is released. Thereby, the identification information of the icon / button at the second contact position TP2 and the relative position of the first contact position TP1 with respect to the second contact position TP2 are determined.

  When the second contact position TP2 occurs, the processor 102, as described above, around the icon button 206, one or more activation items associated with the icon button 205 in relation to the relative position. May be displayed as the activation item display 207. In this case, the second contact position TP2 may be released by being moved on the activation item display 207 of the desired activation item.

  Thus, the first contact position TP1 may be on the icon button 206 of the second contact position TP2. Accordingly, the first contact position TP1 may be formed on a non-icon button area and an arbitrary icon button on the display screen, that is, may be formed at an arbitrary position. There may be only one icon button 206 on the display screen. One icon / button 206 may be displayed on the entire display screen. In this case, the user simply forms the first contact position TP1 and the second contact position TP2 in order at an arbitrary position on the display screen, and the activation item corresponding to the relative position is selected by the processor 102 and activated. Is done.

  In the embodiment, the icon button has been described as an example. However, in the information processing apparatus 10, any other software button can be operated in the multi-touch operation mode in the same manner.

  All examples and conditional expressions given here are intended to help the reader understand the inventions and concepts that have contributed to the promotion of technology, such examples and It is understood that the present invention is not limited to the conditions, and that the organization of such examples in the specification is not related to the superiority or inferiority of the present invention. Although embodiments of the present invention have been described in detail, it will be understood that various changes, substitutions and variations can be made thereto without departing from the spirit and scope of the invention.

(Appendix 1) Obtaining data representing the first contact position and the second contact position on the display screen of the touch panel display device from the touch panel display device,
Determining a relative position of at least one of an orientation and a distance of the first contact position with respect to the second contact position;
Displaying or activating one or more activation items associated with the determined relative position with reference to information representing the activation item associated with the relative position stored in the storage device To decide,
A program for causing an information processing apparatus to execute processing.
(Appendix 2) Further, the information processing apparatus is caused to execute processing for activating one activation item among the one or more activation items,
The process of activating the activation item is responsive to the disappearance of the second contact position while the first contact position is maintained or after the first contact position disappears. The program according to appendix 1 or 2, wherein the program is executed.
(Additional remark 3) The process which determines the said activation item is performed when the said 2nd contact position generate | occur | produces within the limited time after the said 1st contact position lose | disappears, It is characterized by the above-mentioned. The program according to appendix 1 or 2.
(Supplementary note 4) The program according to any one of supplementary notes 1 to 3, further causing the information processing apparatus to execute a process of displaying the determined one or more activation items.
(Supplementary Note 5) Further, the information processing includes a process of activating the one activation item in response to determination or selection of one activation item among the one or more activation items by the user. The program according to appendix 4, wherein the program is executed by a device.
(Supplementary Note 6) In the first operation mode set by the user, the process for determining the activation item is that the second contact position is generated within a limited time after the first contact position disappears. In the second operation mode that is executed by the user and set by the user, the second operation mode is executed when the second contact position occurs at an arbitrary time after the first contact position disappears. The program according to any one of appendices 1 to 5, characterized by:
(Appendix 7) Further, the information processing is caused to execute processing for displaying one or more software buttons associated with each application on the display screen,
The second contact position is a position on one software button among the one or more software buttons on the display screen;
The determined activation item is associated with a combination of the identification information of the one software button of the second touch position and the determined relative position. The program according to any one of appendices 1 to 6.
(Supplementary Note 8) Acquire data representing the first contact position and the second contact position on the display screen of the touch panel display device from the touch panel display device,
Determining a relative position of at least one of an orientation and a distance of the first contact position with respect to the second contact position;
The touch panel determines one or more activation items associated with the determined relative position by referring to information representing the activation item associated with the relative position stored in the storage device A program for causing an information processing device to execute processing to be displayed on a display device.
(Supplementary note 9) Information processing device is
From the touch panel display device, obtain data representing the first contact position and the second contact position on the display screen of the touch panel display device,
Determining a relative position of at least one of an orientation and a distance of the first contact position with respect to the second contact position;
Displaying or activating one or more activation items associated with the determined relative position with reference to information representing the activation item associated with the relative position stored in the storage device To decide,
How to perform processing.
(Appendix 10) a touch panel display device;
A contact position detector that acquires data representing a first contact position and a second contact position on the display screen of the touch panel display device from the touch panel display device;
A relative position determination unit that determines a relative position of at least one of an azimuth and a distance of the first contact position with respect to the second contact position;
Displaying or activating one or more activation items associated with the determined relative position with reference to information representing the activation item associated with the relative position stored in the storage device An activation item determination unit to determine
An information processing apparatus including:

5 Network 10 Information Processing Device 102 Processor 104 Memory 130 Storage Device 110 Touch Panel Display Device 144 Network Interface

Claims (7)

From the touch panel display device, obtain data representing the first contact position and the second contact position on the display screen of the touch panel display device,
Determining the relative position of at least one of orientation and distance of the first contact position relative to the second contact position when the second contact position is on a software button;
One or more activation items associated with the determined relative position are stored with reference to information representing the activation item associated with the relative position with respect to the software button stored in the storage device. , A program for causing an information processing apparatus to execute processing to be determined for display or activation.   The process of activating the activation item is responsive to the disappearance of the second contact position while the first contact position is maintained or after the first contact position disappears. The program according to claim 1, wherein the program is executed.   The process of determining the activation item is performed when the second contact position occurs within a limited time after the first contact position disappears. The program according to claim 1 or 2, characterized by the above.   The process of determining the activation item is executed when the second contact position occurs within a limited time after the first contact position disappears in the first operation mode set by the user. The second operation mode set by the user is executed when the second contact position occurs at an arbitrary time after the first contact position disappears. The program according to any one of claims 1 to 3. From the touch panel display device, obtain data representing the first contact position and the second contact position on the display screen of the touch panel display device,
Determining the relative position of at least one of orientation and distance of the first contact position relative to the second contact position when the second contact position is on a software button;
One or more activation items associated with the determined relative position are stored with reference to information representing the activation item associated with the relative position with respect to the software button stored in the storage device. A program for causing an information processing apparatus to execute processing for determining and displaying on the touch panel display device. Information processing device
From the touch panel display device, obtain data representing the first contact position and the second contact position on the display screen of the touch panel display device,
Determining the relative position of at least one of orientation and distance of the first contact position relative to the second contact position when the second contact position is on a software button;
Referring to information representing one or more activation items associated with a relative position with respect to the software button stored in a storage device, an activation item associated with the determined relative position is determined. How to perform the process of determining to display or activate. A touch panel display device;
A contact position detector that acquires data representing a first contact position and a second contact position on the display screen of the touch panel display device from the touch panel display device;
A relative position determination unit that determines a relative position of at least one of an azimuth and a distance of the first contact position with respect to the second contact position when the second contact position is on a software button;
One or more activation items associated with the determined relative position are stored with reference to information representing the activation item associated with the relative position with respect to the software button stored in the storage device. An activation item determiner that determines to display or activate; and
An information processing apparatus including:

Images