JP2014174764A - Information processing device and control method of the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a GUI technology that improves operability to a display screen to which a touch input can be performed.SOLUTION: A reserved area 408 of a touch operation is set at a location where a finger (thumb) 407 of a user holding a portable terminal device in a single hand reaches out a display screen 402 having a touch sensor mounted. Then, the finger 407 is detached, the display screen 402 is tilted, and display information as whole is moved in a direction 412 corresponding to a direction 411 where a gravity acts. When a button 404 having a touch operation desired is included in the display information during a movement and the button 404 is overlapped with the reserved area 408, the touch operation to the button 404 is executed on behalf of an operator.

Description

  The present invention relates to an information processing apparatus having a display screen capable of touch input and an information input method for the information processing apparatus.

An information processing apparatus that handles a graphical user interface (hereinafter referred to as “GUI”) employs a touch panel display that performs an operation by touching a display screen with a finger or a touch tool (such as a pen). . In particular, in a device such as a smartphone, a tablet, or a portable media player, a user interface (hereinafter referred to as “UI”) operation by touch is the mainstream.
Among them, in particular, touch panel displays mounted on small portable terminals that can be operated with one hand, such as smartphones and portable media players, tend to increase in size in recent years. For this reason, it has become difficult to perform a touch operation with only one hand, for example, a finger does not reach an operation item that can be displayed on the display screen and that allows a UI operation.

  As a technique for solving such a problem, there is a display control method disclosed in Patent Document 1. This display control method is to collect a plurality of links included in a Web page in a designated area and display them on a display screen in a portable electronic device having a touch panel display. Thereby, the link of a web page can be selected easily.

JP 2012-73802 A

  However, in the display control method disclosed in Patent Document 1, when there are many links, the links are reduced and displayed. Further, since links are gathered and displayed together, it is difficult for a user who is an operator to press a desired link. In addition, since the links are collected in the designated area, the layout of the previous Web page is destroyed. Therefore, it is necessary for the user to perform a UI operation after recognizing which link is desired, which is complicated.

  An object of the present invention is to provide an information processing apparatus capable of improving the operability of UI operation on a display screen even when the size of the display screen that can be operated by touch input is large.

  An information processing apparatus according to the present invention, in an information processing apparatus having a display screen equipped with a touch sensor, setting means for setting a reserved area for an operation by touch input in the display screen, and The detection means for detecting the tilt of the display screen, the moving means for moving the display information displayed on the display screen according to the tilt detected by the detecting means, and the display information being moved include the operation An operation execution unit that includes an operation item that activates the process by being executed, and that allows the operation item to be executed when the operation item moves to a position corresponding to the reserved area; It is characterized by having.

  According to the present invention, even when the size of a display screen that can be operated by touch input is large, the operability of UI operation on the display screen can be improved.

The hardware block diagram of the information processing apparatus which concerns on 1st Embodiment of this invention. The function block diagram of the information processing apparatus which concerns on 1st Embodiment. The flowchart which shows the flow of the process in 1st Embodiment. (A)-(e) is the example of a transition of the GUI display in 1st Embodiment. The flowchart which shows the flow of the process in 2nd Embodiment. (A)-(f) is the example of a transition of GUI display in 2nd Embodiment. The flowchart which shows the flow of the process in 3rd Embodiment. (A)-(e) is the example of a transition of GUI display in 3rd Embodiment. The flowchart which shows the flow of the process in 4th Embodiment. (A)-(i) is the example of a transition of the GUI display in 4th Embodiment. The flowchart which shows the flow of the process in 5th Embodiment. (A)-(d) is the example of a transition of GUI display in 5th Embodiment. The flowchart which shows the flow of the process in 6th Embodiment. (A)-(c) is the example of a transition of GUI display in 6th Embodiment.

Hereinafter, exemplary embodiments of the present invention will be described.
[First Embodiment]
FIG. 1 is a hardware configuration diagram of an information processing apparatus capable of inputting information using a GUI. The information processing apparatus 101 includes a controller unit 102, a touch panel display 103, a hard disk (HDD) 104, and an acceleration sensor 108.

  The controller unit 102 is a processor board on which a CPU 105, a ROM 106, a RAM 107, and the like are mounted. The CPU 105 is a processor for comprehensively controlling the operation of the information processing apparatus 101. The ROM 106 stores various programs including a GUI computer program executed by the CPU 105 and data required by these programs. Some programs or data are stored in advance in the HDD 104 which is an external storage device, and are read from the HDD 104 to the RAM 107 as appropriate. A RAM 107 is a work area for the CPU 105 and temporarily stores data used during program execution.

  The touch panel display 103 is a display device having a display screen equipped with a touch sensor, and accepts an operation by a touch input of a user who is an operator, that is, a UI operation. The touch panel display 103 is fixedly provided on a housing (not shown) of the information processing apparatus 101. Therefore, when the information processing apparatus 101 is tilted, the display screen of the touch panel display 103 is tilted in the same manner. In the following description, the “display screen” represents the display screen of the touch panel display 103 included in the information processing apparatus 101.

  On the display screen, a predetermined display layout of a dialog, a frame, and an image constituting the display of an application, a Web page, etc. is displayed as display information. The display information includes operation items such as a button (button image) and a link (URL) that activate a corresponding process when a UI operation is executed. The handling process is a process for starting up an application assigned to an operation item or switching contents to be displayed on the display screen. On the display screen, an input image for accepting an input such as an instruction or designation from the user is appropriately displayed.

  When the user performs a UI operation on the display screen with a finger or a touch tool, operations such as the touched area, the time for which the touch is continued, the number of touches within a predetermined time, the direction operated after the touch, and the operation speed The content is detected by a touch sensor. The area where the display screen is touched with the finger or the touch tool when touched can also be detected as the area of the region. The detection information of the touch sensor is read and used by a touch sensor program stored in the ROM 106. For example, coordinate information representing the position of the area is specified from the touched area. If the identified coordinate information matches the coordinate information of the area where the operation item displayed on the display screen is arranged, it can be recognized that the user has touched the operation item.

  The acceleration sensor 108 is a type of tilt sensor that is fixedly provided at a predetermined part of the information processing apparatus 101. When an external force is applied to the information processing apparatus 101, the acceleration sensor 108 detects the magnitude and direction of the external force. For example, an acceleration around a two-dimensional or three-dimensional axis that occurs when the information processing apparatus 101 is tilted by an external force is output as a detection result. This detection result is processed by a sensor processing program stored in the ROM 106, and is handled as numerical data representing the inclination of the information processing apparatus 101 (the display screen of the touch panel display 103). Instead of the acceleration sensor 108, an angular velocity sensor may be used.

FIG. 2 is a functional configuration diagram of the information processing apparatus 101. The control unit 201 is a function formed when the CPU 105 executes the GUI computer program stored in the ROM 106 in the controller unit 102 of FIG.
The control unit 201 enables reservation of a UI operation and automatic execution of the UI operation in accordance with an input of a detection result of the touch sensor of the touch panel display 103 or the acceleration sensor 108. Specifically, an operation detection unit 202, an operation reservation area setting unit 203, an inclination detection unit 204, a display information movement unit 205, an operation execution determination unit 206, and an operation execution unit 207 that cause the information processing apparatus 101 to be controlled by the control unit 201. The function is made to function as the operation reservation area canceling unit 208.

  The operation detection unit 202 receives the detection result of the touch sensor from the touch panel display 103 and detects the content of the UI operation by the user. For example, an action of touching the display screen of the touch panel display 103 with a finger, an action of keeping the finger touched, an action of moving the touched finger, an action of releasing the finger, and the like are detected. In the following description, an action in which the user touches the display screen of the touch panel display 103 with a finger and immediately releases it is referred to as a “touch operation”, and an action in which the user keeps the finger touched for a certain period of time is referred to as a “long touch operation”. Further, the “display screen” is a display screen of the touch panel display 103.

  The operation reserved area setting unit 203 sets the specified area as a reserved area when a predetermined setting operation is performed on the display screen and an area specified by the operation is specified. The reserved area is an area for reserving execution of subsequent UI operations. In this embodiment, the UI operation to be reserved is a touch operation, and the setting operation is a long touch operation. A UI operation to be reserved and a setting operation can be set by displaying a dialog as an example of an input image on the display screen.

  The operation reserved area setting unit 203 displays on the display screen a dialog for inputting an instruction as to whether or not the reservation area setting is to be accepted, and the reservation area is set when an instruction to set is input from the user. Accepts specification of the setting. Then, it is determined that the reserved area has been designated by the first long touch operation on the display screen after the setting of the reserved area can be accepted, and the area where the long touch operation has been performed is specified as the reserved area.

  The operation reserved area setting unit 203 also displays an operation reservation state indicating that the specified area is set as a reserved area, or the UI operation assigned to the reserved area is actually performed, or Continue on the display screen until the setting is canceled. The size (area) of the reserved area at the time of display may be a predetermined area centered on the touched position or starting from the position, and may be an area where the display screen and the user's finger are in contact with each other. It may be an area. In the latter case, the area of the area is specified by the touch sensor, and coordinate information representing the area of the specified area is held.

  The tilt detection unit 204 detects the tilt of the display screen (including its change speed and change amount) caused by external force being applied to the information processing apparatus 101 after setting the reserved area based on the detection result of the acceleration sensor 108. . Specifically, based on the detection result of the acceleration sensor 108, the change angle (including the change amount and change speed of the angle) of the display screen caused by an external force and the gravity acting on the display screen based on the change angle. Detect direction. There are two-dimensional gradients and three-dimensional gradients. In either case, the direction of gravity is the same.

The display information moving unit 205 moves display information according to the inclination detected by the inclination detecting unit 204 while maintaining the arrangement state (display layout). That is, the entire display information is gradually moved in the direction in which the gravity detected by the inclination detection unit 204 acts. Such a movement mode is referred to as “slide”. For example, when a Web browser is displayed on the display screen, the display information moving unit 205 slides the entire frame of the Web browser together with the contents of the Web page displayed on the Web browser.
The amount of sliding is the amount of change in the inclination detected by the inclination detecting unit 204. The sliding speed can be different depending on the change rate of the slope in a predetermined time. For example, when it is tilted slightly slowly from the horizontal plane, a slight slide amount can be obtained. Note that the sliding speed can be different depending on the positional relationship between the reserved area and the display information. This will be described later.

  The user performs an operation of bringing a specific operation item closer to the reserved area set by the operation reserved area setting unit 203. At this time, the operation execution determination unit 206 monitors whether or not the operation item is arranged in the display information being moved (sliding) and the operation item overlaps the reserved area. And when it overlaps, it is judged whether UI operation (touch operation in this example) allocated to the said operation item is performed.

  Based on the determination result by the operation execution determination unit 206, the operation execution unit 207 enables a UI operation for an operation item overlapping the reserved area on behalf of the user. As execution modes, there are a mode in which automatic execution is performed immediately without waiting for a user instruction, a mode in which automatic execution is performed when a predetermined condition is satisfied, and a mode in which execution is performed after a user instruction is input. In the mode executed after the user's support is input, an input image prompting input of an instruction, for example, a dialog is displayed on the display screen each time.

  The operation reserved area canceling unit 208 cancels the setting of the reserved area set by the operation reserved area setting unit 203. This release is triggered by receiving a notification from the operation execution unit 207 that a UI operation for an operation item has been executed in the reserved area or a detection result of a reservation area release instruction from the operation detecting unit 202.

Next, an example of a processing procedure of the information processing apparatus 101 (control unit 201) will be described with reference to the flowchart of FIG.
When the user performs a UI operation on the display screen, the operation detection unit 202 detects what this UI operation event is (S301). If the detected event is not a long touch operation (NO in S302), the operation reservation area setting unit 203 returns to S301. That is, the state before the event detection of the UI operation is restored.
If the event is a long touch operation (S302: YES), the user intends to set a reserved area for the UI operation. Therefore, the operation reserved area setting unit 203 detects the position where the long touch has been made (S303), and sets the detected position as the reserved area (S304). Note that the reserved area has a predetermined area as described above. The operation reservation area setting unit 203 also starts displaying an operation reservation state indicating that a UI operation reservation area is set on the display screen (S305).

The inclination detection unit 204 determines whether or not the information processing apparatus 101 is inclined from the time when the reservation area is set and the display of the operation reservation state is started (S306). When it is determined that the display is tilted (S306: YES), the display information moving unit 205 slides the display information according to the tilt (S307). The sliding direction is the direction in which gravity works on the display screen. At this time, the entire display information slides while maintaining the display layout. During the slide, the operation execution determination unit 206 determines whether or not the operation item included in the display information overlaps the reserved area (S308). If they do not overlap (S308: NO), the process returns to S307. That is, the slide is continued.
If they overlap (S308: YES), the operation execution unit 207 automatically executes a UI operation for the overlapped operation items, in this example, a touch operation (S309).

In S306, when it is determined that the information processing apparatus 101 is not tilted (S306: NO), the operation reservation area releasing unit 208 determines whether or not to end the display of the operation reservation state (S310). If not finished (NO in S310), the process returns to S306. That is, the inclination detection is continued. As described above, it is determined that the display of the operation reservation state is to be ended, for example, when a touch operation is performed in the reserved area, or when the user instructs to cancel the setting of the reserved area.
When the display of the operation reservation state is to be ended (S310: YES), the operation reservation area releasing unit 208 ends the display of the operation reservation state (S311). Further, the reserved area set in S304 is canceled (S312). The information processing apparatus 101 enables the setting of the reserved area for the display screen and the execution of the UI operation for the operation item according to the above flow.

Next, a transition example of GUI display corresponding to the process shown in FIG. 3 will be described with reference to FIG. FIG. 4 shows an example in which a portable terminal device 401 having a touch panel display 402 is used as the information processing device 101.
Referring to FIG. 4A, a Web browser application for displaying a Web page is displayed on a full screen on the display screen. Display information such as an address bar 403, a button 404, a link 405, and an image 406 is displayed on the Web page. The address bar 403 is for inputting Internet address information (URL). A button 404 is used by the user to activate a corresponding process by a touch operation. A link 405 is used to make a transition to another Web page. These can all be operation items to be reserved for UI operation.
The user is holding the portable terminal device 401 with one hand (left hand) and is about to perform a touch operation with a finger (thumb) 407 on the display screen. At this time, the user wants to perform a touch operation on the button 404, but the finger 407 of the hand does not reach the button 404, and the touch operation cannot be performed. Therefore, the user performs UI operations as shown in FIGS. 4B to 4E.

FIG. 4B shows a state in which the user performs a long touch operation on an arbitrary position near the button 404 when the finger 407 of the handle reaches. This is the first UI operation performed by the user after setting the reserved area. Thereafter, when the user releases the finger 407 from the display screen as shown in FIG. 4C, the position where the long touch operation is performed on the display screen is set as the reserved area 408.
As shown in FIG. 4D when the mobile terminal device 401 is viewed from below, it is assumed that the user tilts the mobile terminal device 401 from the state 409 to the state 410 in order to bring the button 404 closer to the reserved area. As a result, the display screen is tilted in the direction 411 in which gravity works. Then, as shown in FIG. 4E, the Web browser application 413 that has been displayed so far slides in the same direction 412 as the direction 411 in which gravity works while maintaining the display layout. When the button 404 approaches the reserved area 408 by this slide and eventually overlaps, the touch operation on the button 404 is automatically executed on behalf of the user.
In the example of FIG. 4E, the remaining display portion 414 after sliding is displayed in a single color, but another application that is being activated may be displayed in this display portion 414. Alternatively, the web browser application 413 may be displayed again.

  With such a UI operation, even when the size of the display screen is large, the user can activate the corresponding process by operating only the thumb of the hand while holding it with one hand. Therefore, the operability at the time of touch input can be significantly improved.

[Second Embodiment]
In the second embodiment, an example will be described in which the user continues a long touch operation on the reserved area (hereinafter, this state is referred to as a “long touch state”). In addition, the description of the part which overlaps with description of 1st Embodiment is abbreviate | omitted.
FIG. 5 is a flowchart showing the flow of processing in the second embodiment. Portions overlapping those in FIG. 3 are omitted. FIG. 5 shows the flow of processing after it is determined in FIG. 3 that the information processing apparatus 101 is tilted (S306: YES). At this time, the reserved area has already been set by the long touch operation.

Since the information processing apparatus 101 is tilted, the display information moving unit 205 determines whether the reserved area is in the long touch state based on the detection result of the operation detecting unit 202 (S501). When not in the long touch state (S501: NO), the processing flow is the same as that in the first embodiment (FIG. 3: S307 to S309).
When it is determined that the state is the long touch state (S501: YES), the display information moving unit 205 slides the entire displayed display information in the direction in which gravity works in accordance with the tilt of the display screen (S502). Meanwhile, the operation execution determination unit 206 determines whether or not the long touch state has been released (S503). That is, it is determined whether the long touch state has been released by releasing the user's finger from the display screen while the display information is being slid. If not canceled (S503: NO), the process returns to S502. That is, the slide is continued.

When it is determined that the long touch state has been released (S503: YES), the operation execution determination unit 206 determines whether or not the operation item overlaps the reserved area (S504). If they do not overlap (S504: NO), the routine of FIG. 5 is exited and the process proceeds to S310 of FIG. That is, it is determined whether or not to end the operation reservation state.
If the operation item overlaps the reserved area (S504: YES), it means that the operation item overlaps the reserved area when the user's finger is released. In this case, the operation execution unit 207 automatically executes a UI operation for the operation item, that is, a touch operation (S309). Thereafter, the routine of FIG. 5 is exited and the process proceeds to S310 of FIG.

A transition example of GUI display corresponding to the processing shown in FIG. 5 will be described with reference to FIG. FIG. 6 shows an example in which a portable terminal device 401 having a touch panel display 402 is used as the information processing apparatus 101, as in FIG. The same elements as those in FIG. 4 are denoted by the same reference numerals.
6A has the same contents as FIG. 4A, and FIG. 6B has the same contents as FIG. 4B. FIG. 6C shows a state in which the user keeps pushing the reserved area 408 while keeping the finger 407 away from the display screen and in a long touch state. In this state, it is assumed that the mobile terminal device 401 is tilted from the state 409 to the state 410 as shown in FIG. In the case of FIG. 6D, the display screen is tilted in the direction 411 in which gravity acts. Accordingly, as shown in FIG. 6E, in response to the tilt, the Web browser application 413 slides in a direction 412 corresponding to the direction in which gravity works while maintaining the display layout. During the slide, the finger 407 maintains a long touch state. Thereafter, as shown in FIG. 6 (f), the reserved area 408 and the button 404 overlap with each other, and at the same time, the user releases the finger 407. Then, a touch operation on the button 404 is executed with the above-described operation of the user as a trigger.

  As described above, in the second embodiment, when the operation state other than the operation reserved for execution in the reserved area is continued and the operation state is canceled when the operation item overlaps the reserved area, Perform reserved operations immediately. That is, when the UI operation reserved for the operation item is different from the UI operation for setting the reservation area, the UI operation for the operation item is automatically executed when the operation item overlaps the reservation area. The Even in such a form, the same effect as the first embodiment can be obtained.

[Third Embodiment]
In the third embodiment, an example in which a reserved area is newly set at a position where an operation item is arranged will be described. Note that a description overlapping with the description of the first embodiment is omitted.
FIG. 7 is a flowchart showing the flow of processing according to the third embodiment, and shows the flow of processing performed between S301 and S305 in FIG.

The process (S301 to S303) until the long touch position is detected by the user is the same as the process (FIG. 3) of the first embodiment. In the third embodiment, when the long touch position is detected, the operation reservation area setting unit 203 determines whether an operation item is arranged at the detected long touch position (S701). If it is not arranged (S701: NO), the processing is the same as in the first embodiment, so the process proceeds to S305 in FIG.
If it is arranged (S701: YES), the finger has reached the operation item in the first place. However, when there are a plurality of operation items, the operation item that the finger has reached may not actually be an operation item for which a UI operation is desired.
Therefore, in the third embodiment, whether the operation reservation area setting unit 203 executes a UI operation for an operation item that overlaps the reservation area, or sets its position as a reservation area for another operation item. Display a dialog about whether or not. Then, when an instruction to set the reserved area is input, the long touch position is set as the reserved area.

Next, a transition example of GUI display according to the third embodiment will be described with reference to FIG. FIG. 8 is partially the same as FIG. The same elements as those in FIG. 4 are denoted by the same reference numerals.
In FIG. 8A, a button B801, which is another example of an operation item, is added to the example of the Web page shown in FIG. As shown in FIG. 8B, the user performs a long touch operation on the button B 801, but the UI operation that the user wants to perform is a touch operation on the button 404. In this case, as shown in FIG. 8C, the mobile terminal device 401 performs an operation of pressing the button B801 for which the long touch operation has been performed, or sets the position of the button B801 for which the long touch operation has been performed as a reserved area. A dialog 803 for selecting whether or not to set is displayed. As shown in FIG. 8D, when the reserved area setting 804 is selected, as shown in FIG. 8E, a reserved area 805 for executing a UI operation on the button 404 is displayed at the position of the button B 801. Is set.

  As described above, according to the third embodiment, when it is not desired to execute the UI operation for the operation item even though the operation item overlaps the reserved area, the operation item is newly added at the position of the operation item. A reserved area can be set. Therefore, in addition to the effect of the first embodiment, an effect that it is possible to realize a UI operation environment that more closely matches the user's intention is obtained.

[Fourth Embodiment]
In the fourth embodiment, a technique for superimposing an operation item desired by a user on a reserved area will be described. Note that a description of the same parts as those in the first embodiment is omitted.
FIG. 9 is a flowchart showing a flow of processing according to the fourth embodiment, and shows a flow of processing after YES is determined in the processing of S306 of FIG. In the fourth embodiment, the description of the same parts as those in the first embodiment is omitted.
In the process of sliding the display information according to the inclination of the information processing apparatus 101 (S307), if it is determined that the operation item has overlapped with the reserved area (S308: YES), the operation execution determining unit 206 displays the overlapped operation item. It is determined whether or not an operation can be performed on (S901). If it can be executed (S901: YES), the process is the same as that of the first embodiment, so the routine of FIG. 9 is exited and the process proceeds to S309 of FIG. If not executed (S901: NO), the process returns to S307.

  For example, whether or not an operation item can be executed is determined by displaying a dialog for selecting whether to perform an operation on the operation item overlapping the reserved area in the vicinity of the reserved area, that is, in an area where the user's finger can reach. To select. Or you may perform the determination of the feasibility performed by estimation of a user's intention. The estimation is performed based on, for example, whether or not the state in which the information processing apparatus 101 is tilted continues for a predetermined time or more in a state where the operation item overlaps the reserved area. If it continues, it is estimated that the user does not want the operation, and the operation is not executed.

Next, a transition example of GUI display according to the fourth embodiment will be described with reference to FIG. FIG. 10 is partially the same as FIG.
In FIG. 10A, a link (Link2) 1001 that is an example of an operation item is newly added to the example of the Web page shown in FIG. As shown in FIG. 10B, after the user's left thumb 407 reaches and a long touch operation is performed on an arbitrary position close to the link (Link2) 1001, as shown in FIG. Then, the thumb 407 is released from the display screen. Thereby, the position where the long touch operation is performed is set as the reserved area 408. In this state, it is assumed that the user tilts the portable terminal device 401 from the horizontal state 409 to the state 410 as shown in FIG. In this case, the direction 411 is the direction in which gravity works. Then, as shown in FIG. 10E, the Web browser application 413 displayed on the display screen slides in the direction 412 in which gravity works according to the tilt while maintaining the display layout, and the link (Link2 ) 1001 overlaps the reserved area 408.

  At this time, as shown in FIG. 10 (f), a dialog 1003 is displayed for allowing the user to select whether or not to execute the link (Link2) 1001 overlapping the reserved area 408. As shown in FIG. 10G, when the user selects 1004 which is not executed in the dialog 1003, the touch operation on the link (Link2) 1001 is not executed. Subsequently, as shown in FIG. 10 (h), the inclination is continued, and as shown in FIG. 10 (i), the slide in the direction 412 in which the gravity works is continued. As a result, the user desires the reserved area 408 eventually. Buttons 404 that are operation items are overlapped, and operations on the buttons 404 can be executed. As a result, only the operation on the button 404 can be executed without executing the operation of the link (Link2) 1001.

If the slide of the display information continues after the link (Link2) 1001 overlaps the reserved area 408, the user desires to operate an operation item other than the overlapping link (Link2) 1001. It is assumed that Therefore, in this case, the display of FIGS. 10F and 10G may be omitted and the slide may be resumed so that the button 404 overlaps the reserved area 408 as shown in FIG.
As described above, according to the fourth embodiment, it is possible to overlap only the operation items desired by the user and execute only the operation on the operation items.

[Fifth Embodiment]
In the fifth embodiment, another control method of slide movement will be described. Note that the description of the same parts as those in the first embodiment is omitted. FIG. 11 is a flowchart showing the flow of processing in the fifth embodiment, and shows the flow of processing after YES is determined in step S306 in FIG.

  When the tilt of the information processing apparatus 101 is detected after setting the reserved area, the display information moving unit 205 analyzes the position of the reserved area and the operation item (S1101). Specifically, the display information moving unit 205 determines whether there is an operation item in the direction opposite to the direction in which the information processing apparatus 101 is tilted and gravity works from the reserved area (S1102). If there is an operation item (S1102: YES), the display information moving unit 205 is in a direction opposite to the direction in which gravity works from the reserved area, and until the operation item with the closest distance from the reserved area overlaps. The moving speed (slide speed) of the display information is accelerated (S1103). When there is no operation item (S1102: NO), the display information moving unit 205 slides the display information at a predetermined constant speed (S1104).

Next, a transition example of GUI display according to the fifth embodiment will be described with reference to FIG. FIG. 12 is partially the same as FIG.
In FIG. 12A, a reserved area 1201 is set to the same contents as the information on the mobile terminal device 401 and the Web page shown in FIG. Assume that the user holds only the mobile terminal device 401 with one hand (left hand) and tilts only the upper part of the mobile terminal device 401 to the left as shown in FIG. As shown in FIG. 12C, the button 404 that is an operation item is arranged in a direction 1203 that advances from the reserved area 1201 in a direction opposite to the direction 1202 in which gravity acts. Then, as shown in FIG. 12 (d), the button 404 moves faster than usual in the direction of gravity 1204 along with the entire web browser application 1205, and eventually overlaps the reserved area 1201. Thereby, an operation for the button 404 is executed.
As described above, in the fifth embodiment, the mobile terminal device 401 is tilted and the display information displayed on the display screen of the touch panel display 402 is accelerated and moved. However, the mobile terminal device 401 may be switched by setting the mode, and the display information may be accelerated in a certain mode and moved at a constant speed in another mode.

[Sixth Embodiment]
In the sixth embodiment, an example will be described in which when an operation on an operation item that overlaps a reserved area is executed, the operation item requires two or more operations. FIG. 13 is a flowchart showing a process flow according to the sixth embodiment, and shows a process flow after the process of S309 of FIG.
When the operation item overlaps the reserved area, the operation execution unit 207 determines whether or not two or more operations are required when executing the operation on the operation item (S1301). For example, an HTML select box is a pull-down menu, but when the select box is touched, a menu including other selection candidates is displayed in a pull-down menu. In such a case, it is determined that two or more operations are required when performing an operation on the operation item.

If two or more operations are not required (S1301: NO), the process is the same as in the first embodiment, so the process exits the routine of FIG. 13 and proceeds to S310 of FIG.
When two or more operations are required (S1301: YES), the operation execution unit 207 waits for the second and subsequent operations by the user on the operation item. When the second and subsequent operations are performed, a corresponding process, for example, a process of displaying a pull-down is executed (S1302). The operation execution unit 207 determines whether all operations after the second operation item have been executed (S1303). If all the processes have not been executed (S1303: NO), the process returns to S1302. If all the processes have been executed (S1303: YES), the process exits the routine of FIG. 13 and proceeds to S310 of FIG.

Next, an example of GUI display transition according to the sixth embodiment will be described. FIG. 14 is partially the same as FIG. 4 shown in the first embodiment, and thus the description of the overlapping parts is omitted. The same reference numerals are used for portions having the same contents as in FIG.
In FIG. 14A, in the Web page shown in FIG. 4A, a select box 1401 which is an operation item is newly added and displayed, and a reserved area 1402 is set. When the mobile terminal device 401 is tilted, as shown in FIG. 14B, the select box 1401 that can be touch-operated approaches the reserved area 1402. Eventually, when the select box 1401 overlaps the reserved area 1402 in the area 1403, as shown in FIG. 4C, an operation for the overlapped select box 1401 is automatically executed, and other selection items are displayed in the pull-down 1404. That is, the second and subsequent operations for the touch-operable eye are accepted. The user can select a desired item from other selection items, for example, by a touch operation.

  As described above, even when the operation item overlapped with the reserved area requires execution of two or more operations, the user can move near the position where the first operation is automatically executed (the position of the reserved area). Second and subsequent UI operations can be performed.

[Seventh Embodiment]
In the seventh embodiment, another setting method for a reserved area will be described. In the embodiments so far, the operation reservation area setting unit 203 has been described as setting the position where the user first performed a long touch operation on the display screen as the reservation area. The function that enables such setting may be prepared by default, but may be activated by mode switching in advance.
In the latter case, the operation reservation area setting unit 203 causes the display screen to display a dialog for starting a reservation area setting mode indicating a state in which the reservation area is set from the normal UI operation mode. In the reserved area setting mode, the setting is activated, and it is possible to set contents such that the user first sets the touch position on the display screen or sets the reservation operation as a touch operation. In this way, by making it possible to set the reserved area by switching the mode in advance, the reserved area can be set only when such a UI operation is required.

[Modification]
In the first to seventh embodiments, as an example of the information processing apparatus, a mobile terminal apparatus having a display screen that can be grasped with one hand and can be operated with a finger of the grasped hand has been described. Information processing to which the present invention is applied The apparatus is not necessarily limited to the above example. For example, the present invention can be implemented even when a relatively large tablet personal computer is used.

Further, the present invention can be implemented even if the touch panel display is an information processing apparatus that is separated and independent. However, in this case, the acceleration sensor 108 is provided on the touch panel display.
In each embodiment, the reservation area reserves a touch operation, and the setting is performed by a long touch operation. However, the content of the UI operation is not limited to such an operation. . Furthermore, in each embodiment, although it demonstrated on the assumption that a user operates with the finger | toe 407, it can also implement in the aspect input by touch with a touch tool or other tools.

Claims (13)

In an information processing apparatus having a display screen equipped with a touch sensor,
Setting means for setting a reservation area for operations by touch input on the display screen;
Detecting means for detecting an inclination of the display screen after the reserved area is set;
Moving means for moving the display information displayed on the display screen in accordance with the inclination detected by the detecting means;
The display information being moved includes an operation item that activates processing when the operation is executed, and when the operation item is moved to a position corresponding to the reserved area, the operation item is displayed. An operation execution means for enabling an operation, and
Information processing device. The setting means is configured to set the specified area as the reserved area when a predetermined setting operation is performed on the display screen and an area specified by the operation is specified. To
The information processing apparatus according to claim 1. The setting means sets, as the reserved area, an area designated by an initial touch input to the display screen after the reservation area can be accepted.
The information processing apparatus according to claim 2. The setting means can set a new reserved area for another operation item instead of executing an operation for the operation item when the operation item overlaps the reserved area after setting the reserved area. It is characterized by
The information processing apparatus according to claim 2. The setting means is characterized in that the display that the reserved area is set is continued on the display screen until an operation on the reserved area is executed or the setting is canceled.
The information processing apparatus according to claim 2, 3 or 4. The reserved area has an area where an operator's finger or touch tool when touch-input is in contact with the display screen,
The information processing apparatus according to any one of claims 1 to 5. The detection means includes an inclination sensor that detects the magnitude and direction of an external force applied to the display screen, and a change in a change angle from a horizontal plane of the display screen by the external force based on a detection result of the inclination sensor and its change. Detecting the direction in which gravity acts on the display screen according to a change in angle;
The moving means moves the entire display information in the direction detected by the detecting means while maintaining the display layout.
The information processing apparatus according to claim 1. The moving means accelerates the moving speed of the display information when the operation items included in the display information are arranged in a direction opposite to the direction of the reserved area.
The information processing apparatus according to claim 7. The operation execution means continues the operation state other than the operation reserved for execution in the reserved area, and the reservation is performed when the operation state is canceled when the operation item overlaps the reserved area. Performing the performed operation immediately,
The information processing apparatus according to claim 1. When the operation information is continuously moved even if the operation item overlaps the reserved area, the operation execution unit is configured to allow the operator to select whether or not to execute before performing the operation on the operation item. An image is displayed in the vicinity of the reserved area, and the operation is executed when execution is selected.
The information processing apparatus according to claim 1. When the operation item overlapped with the reserved area requires execution of the operation twice or more, the operation executing means executes the first operation on the operation item, and then the second and subsequent times near the reserved area. Forming an area for receiving an input of the operation,
The information processing apparatus according to claim 1. An information processing apparatus having a display screen on which a touch sensor is mounted and an inclination sensor,
Setting means for setting a reservation area for operations by touch input on the display screen;
Detecting means for detecting an inclination of the display screen after the reserved area is set based on a detection result of the inclination sensor;
Moving means for moving display information displayed on the display screen according to the inclination detected by the detecting means;
The display information being moved includes an operation item that activates processing when the operation is executed, and when the operation item is moved to a position corresponding to the reserved area, the operation item is displayed. Function as an operation execution means for making the operation executable,
Computer program. A method executed by an information processing apparatus having a display screen on which a touch sensor is mounted,
After setting a reserved area for operation by touch input on the display screen, the tilt of the display screen is detected based on the detection result of a predetermined tilt sensor, and the display displayed on the display screen according to the tilt When the display item being moved includes an operation item that activates processing by executing the operation, and the operation item has moved to a position corresponding to the reserved area , Wherein the operation for the operation item can be executed,
A method for inputting information to an information processing apparatus.