JP2015158758A - Image display processing device, image display processing method and image display processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To attain an operation input that facilitates an adjustment of a display image to an intended state.SOLUTION: A contact location is detected that has one or more points on a display unit 105D by an indication body such as a finger and the like through a touch pad 105P. A gesture determination unit 110 is configured to sequentially receive detection results to be provided from the touch pad 105P to determine how an operation is conducted through the indication body on the basis of a change in respective contact positions of the indication body causing at least two points on the display unit 105D to be contacted. A mode switch unit 111 is configured to switch to a processing mode according to the determination result of the gesture determination unit 110, and display control means is configured to perform a notification display of the processing mode, and change a display mode of the display image according to the processing mode on the basis of a drag operation to the display unit 105D.

Description

  The present invention relates to a device including a so-called touch panel configured by combining a display device and a position input device, a method and a program used in the device.

  In recent years, various inventions have been made in the technical field relating to a user interface using a touch panel configured by combining a display device such as a liquid crystal panel and a position input device such as a touch pad. For example, Patent Document 1 described later discloses an invention related to an electronic book including a display unit capable of displaying a map image. Patent Document 1 describes that a map image enlargement instruction and an enlargement amount can be input by an operation (pinch-out operation) of moving two fingers in contact with a touch panel away from each other. Patent Document 1 describes that a map image reduction instruction and a reduction amount can be input by an operation (pinch-in operation) in which two fingers brought into contact with a touch panel are brought close to each other. Further, Patent Document 1 describes that a rotation instruction and a rotation amount of a map image can be input by a rotation operation in which two fingers are brought into contact with a touch panel and another finger is rotated around one finger. Has been.

JP 2013-061950 A

  As described in Patent Document 1 described above, the pinch-out operation, the pinch-in operation, and the rotation operation are easy-to-understand operations that are intuitively linked to the process to be executed. The quantity can be entered at the same time. For this reason, even if the user does not confirm the operation of the device through an operation manual or the like, processing such as image enlargement, reduction, and rotation can be performed by the user's intention by easy-to-understand operations such as pinch-out operation, pinch-in operation, and rotation operation. Can be done easily depending on the case.

  However, when it is desired to enlarge the image beyond the range that can be enlarged by a single pinch-out operation, the finger must be removed from the touch panel and the pinch-out operation must be performed again. The same applies to image reduction or rotation. If the display image does not reach the target state by one pinch-in operation or one rotation operation, another pinch-in operation or rotation operation is performed. There is a need.

  As described above, in the case of so-called pinch-out operation, pinch-in operation, and rotation operation that have been conventionally used, the same operation may have to be performed a plurality of times before the display image is brought into a target state. However, it may take time to adjust the display image. For this reason, there exists a subject that provision of the new operation input which is easier to understand and is easy to adjust to the state which made the display image the objective is desired. In particular, in mobile terminals that are often operated with one hand, it is important to solve the problem.

  In view of the above points, an object of the present invention is to enable an operation input that can be easily adjusted to a state in which a display image is intended.

In order to solve the above problem, an image display processing device according to claim 1 of the present application is provided.
Contact position detecting means for detecting the one or more contact positions by an indicator that is in contact with one or more positions on a display unit on which an image is displayed;
The detection result from the contact position detecting means is sequentially received, and based on the change in the contact position of each indicator that is in contact with at least two points on the display unit, the indicator is passed through the indicator on the display unit. A discriminating means for discriminating an operation performed on the device;
Mode switching means for switching to a processing mode corresponding to the operation determined by the determination means;
Processing mode display processing means for performing display on the display unit to notify the processing mode switched by the mode switching means;
After the processing mode is switched by the mode switching unit, the display unit according to the operation performed by the user determined based on the detection result from the contact position detection unit and the switched processing mode And display control means for changing the display mode of the image displayed on the screen.

  According to the image display processing apparatus of the first aspect of the present application, the contact position detecting means detects one or more contact positions on the display unit by an indicator such as a finger. The discriminating means sequentially receives the detection results from the contact position detecting means, and changes the contact positions of the indicators brought into contact with at least two points on the display portion by the indicators to the display portion. Determine what operation was performed.

  The mode switching means switches to the processing mode corresponding to the operation determined by the determination means. The processing mode display processing means performs display on the display unit to notify the user of the switched processing mode. Then, when an operation is performed on the display unit by the user, the display control unit changes the display mode of the image displayed on the display unit according to the operation and the current processing mode.

  Thereby, for example, the processing mode is switched by a so-called finger gesture such as a pinch-out operation, a pinch-in operation, and a rotation operation performed by bringing two fingers (two indicators) of the user into contact with at least two points on the display unit. be able to. Thereafter, for example, the display mode of the display image can be adjusted according to the processing mode by a simple operation such as a tap operation or a drag operation. Therefore, it is possible to realize an operation input that can easily adjust the display image to a target state.

  Note that so-called finger gestures performed by bringing an indicator such as a user's finger into contact with at least two points on the display unit include a pinch-in operation, a pinch-out operation, and a rotate operation. The pinch-in operation is an operation in which two indicators such as a finger are brought into contact with different positions on the display unit, and one or both of the indicators are moved to reduce the distance between the contact positions of the two indicators. In addition, the pinch-out operation is an operation in which two pointers such as fingers are brought into contact with different positions on the display unit, and one or both pointers are moved to increase the distance between the contact positions of the two pointers. is there. The rotate operation is an operation in which two indicators such as a finger are brought into contact with different positions on the display unit, and one or both of the two indicators are moved to draw a circle.

  According to the present invention, it is possible to realize an operation input that can be easily adjusted to a target state of a display image.

It is a block diagram for demonstrating the structural example of the portable terminal 1 of embodiment. It is a figure for demonstrating the process when pinch-in operation is performed. It is a figure for demonstrating the process when pinch out operation is performed. It is a figure for demonstrating the process when rotation operation is performed. 5 is a flowchart for explaining a typical example of an operation input receiving process in the mobile terminal 1; It is a flowchart for demonstrating reduction mode processing. It is a figure for demonstrating the example of a display at the time of reduction mode in case a display image is a map image. It is a flowchart for demonstrating expansion mode processing. It is a flowchart for demonstrating a rotation mode process. It is a figure for demonstrating the example of a display at the time of rotation mode in case a display image is a map image.

  Hereinafter, embodiments of the apparatus, method, and program of the present invention will be described with reference to the drawings. In recent years, various electronic devices having a touch panel configured by combining a display device such as a liquid crystal panel and a position input device such as a touch pad have been widely used. For example, a high-function mobile phone terminal called a smartphone or a tablet PC (PC is an abbreviation for Personal Computer) is a typical electronic device provided with a touch panel.

  Also, various personal computers are provided with touch panels. Conventionally, guidance information display devices provided in sightseeing spots, ATMs (automated teller machines) such as banks, CDs (cash dispensers), ticket vending machines such as stations, etc., are often equipped with touch panels. .

  The present invention is applicable to various electronic devices provided with the touch panel as described above. However, in the embodiment described below, in order to simplify the description, an example in which the present invention is applied to a high-function mobile phone terminal called a smartphone that has been widely used in recent years will be described. .

[Configuration example of high-function mobile phone terminal]
FIG. 1 is a block diagram for explaining a configuration example of a high-function mobile phone terminal (hereinafter referred to as a mobile terminal) 1 according to this embodiment. In FIG. 1, a wireless communication unit 101, a transmission / reception antenna 101A, a control unit 102, a storage device 103, a GPS unit 104, and a GPS antenna 104A are fundamental to the portable terminal 1 such as a wireless communication function, an information processing function, and a current position acquisition function. This is a component for realizing various functions.

  The display unit 105D and the touch pad 105P constitute a touch panel 106 and realize an image and video display function and an operation input reception function. The display unit 105D is a thin display device such as a liquid crystal panel, and has a relatively large screen size of 4 inches or more. The touch pad 105P is a position input device, and is provided over the entire display area of the display unit 105D. Even when a plurality of indicators such as fingers touch the display unit 105D at the same time, the plurality of contact positions are provided. Each of these can be detected.

  Accordingly, a pinch-in operation, a pinch-out operation, a rotate operation, and the like can be received through the touch panel 106, and a function as a so-called software key can be realized along with information displayed on the display unit 105D. Of course, an operation input by one indicator such as a tap operation, a drag operation, and a flick operation can be accepted through the touch panel 106.

  In the mobile terminal 1 shown in FIG. 1, the gesture discriminating unit 110, the mode switching unit 111, and the display control unit 112 are main parts for realizing the present invention. Based on information (coordinate information) indicating the contact position on the display unit 105D by an indicator such as a user's finger, which is detected at a predetermined timing through the touch pad 105P, the gesture discriminating unit 110 uses the indicator. To determine if an operation was performed. The gesture determination unit 110 notifies the determination result to the mode switching unit 111 and the display control unit 112.

  For example, it is assumed that an indicator such as a user's finger touches two distant points on the display unit 105D. In this case, the gesture determination unit 110 sequentially identifies the two contact positions and the distance between the two contact positions from the information indicating the two contact positions sequentially supplied from the touch pad 105P. . Based on the contact position and the change state thereof, the distance between the two contact positions and the change state thereof, the gesture determination unit 110 performs an operation (finger gesture) performed on the display unit 105D. Determine something.

  For example, when an indicator such as a finger contacts two distant points on the display unit 105D, the contact position of the two points gradually moves, and the distance between the two points decreases with time. To do. In this case, the gesture determination unit 110 determines that the operation performed on the display unit 105D is a pinch-in operation, and notifies the mode switching unit 111 of the determination result. In addition, when an indicator such as a finger contacts two distant points on the display unit 105D, the contact position of the two points gradually moves, and the distance between the two points increases with time. To do. In this case, the gesture determination unit 110 determines that the operation performed on the display unit 105D is a pinch-out operation, and notifies the mode switching unit 111 of the determination result.

  In addition, an indicator such as a finger contacts two distant points on the display unit 105D, and the distance between the two contact positions does not change so much, but one or both of the contact positions may change with time. Suppose you are moving in a circle. In this case, the gesture determination unit 110 determines that the operation performed on the display unit 105D is a rotation operation, and notifies the mode switching unit 111 of the determination result. Here, the movement so as to draw a circle is roughly the following operation. That is, when one contact position moves around the center of the other contact position, or two points of contact centering on or near the midpoint of a straight line connecting the two contact positions When the position moves around the center, the movement means drawing a circle.

  Of course, the gesture determination unit 110 can also determine a tap operation, a drag operation, a flick operation, a long press operation, and the like, and can notify the determination result to the display control unit 112, for example. Note that the tap operation is an operation in which an indicator such as a finger is brought into contact with one point on the display unit 105D very briefly. The drag operation is a so-called drag operation that changes the contact position by moving an indicator such as a finger in contact with one point on the display unit 105D. In addition, the flick operation is an operation of quickly sliding the indicator such as a finger that is in contact with one point on the display unit 105D only slightly or quickly catching (playing) it in one direction. The long press operation is an operation of maintaining an indicator such as a finger in contact with one point for a while (for example, a few seconds) without moving the contact position.

  The mode switching unit 111 switches the display processing mode of the own device according to the determination result of the gesture determination unit 110 when an image is displayed on the display unit 105D. That is, when the gesture determining unit 110 determines that the pinch-in operation has been performed, the mode switching unit 111 controls the display control unit 112 to set the display processing mode of the own device to the reduced mode. Further, when the gesture determining unit 110 determines that the pinch-out operation has been performed, the mode switching unit 111 controls the display control unit 112 so that the display processing mode of the own device is set to the enlarged mode. Further, when the gesture determining unit 110 determines that the rotation operation has been performed, the mode switching unit 111 controls the display control unit 112 so that the display processing mode of the own device is set to the rotation mode.

  Furthermore, when the mode switching unit 111 receives notification from the gesture determination unit 110 that a predetermined mode end operation has been performed after switching to each processing mode described above, the mode switching unit 111 switches from each processing mode to the normal display processing mode. The display control unit 112 is controlled so that Here, the normal display processing mode is a mode in which the display image can be used by fixing the reduction ratio, the enlargement ratio, the orientation, and the like of the display image, for example, by allowing the display image to be scrolled.

  The display control unit 112 performs a process of displaying display information indicating what the selected display processing mode is on the display unit 105D. Further, the display control unit 112 performs a process of adjusting and redisplaying the display image according to the selected display processing mode and a predetermined adjustment instruction operation such as a drag operation or a tap operation on the display unit 105D. . Specifically, reduction processing is performed when the display processing mode is the reduction mode, enlargement processing is performed when the display processing mode is the enlargement mode, and rotation processing is performed when the display processing mode is the rotation mode. The display control unit 112 also performs various processes related to the display image, such as a scrolling process of the display image, in response to an operation input from the user.

  2-4 is a figure for demonstrating concretely the main processes which the gesture discrimination | determination part 110, the mode switching part 111, and the display control part 112 perform. 2 shows a case where a pinch-in operation is performed on the display unit 105D, FIG. 3 shows a case where a pinch-out operation is performed on the display unit 105D, and FIG. 4 shows a case where the display unit 105D is rotated. It is a figure for demonstrating the case where operation is performed.

  The gesture determination unit 110, the mode switching unit 111, and the display control unit 112 function when a display image is displayed on the display unit 105D. Therefore, in the following description, it is assumed that some image such as a map, characters (text information), a still image, and a moving image is displayed on the display unit 105D shown in FIGS. To do.

  First, a case where a pinch-in operation is performed on the display unit 105D will be described with reference to FIG. As shown in FIG. 2A, two fingers (for example, thumb and forefinger) of the user are brought into contact with the display portion 105D, and the two fingers are maintained as shown by a large arrow while maintaining the contact state. Suppose that a pinch-in operation that brings In this case, the gesture determination unit 110 determines that a pinch-in operation has been performed on the display unit 105D based on a change in coordinate information indicating the contact positions of two fingers sequentially provided from the touch pad 105P. This is notified to the mode switching unit 111.

  When the mode switching unit 111 receives a notification from the gesture determination unit 110 that a pinch-in operation has been performed, the mode switching unit 111 forms a control signal for switching to the reduction mode and supplies this to the display control unit 112. The display control unit 112 first forms a mode display H1 indicating that the own device has been switched to the reduction mode, and displays this on the upper right end of the display unit 105D as shown in FIG. Thereafter, the display control unit 112 performs a display control process by recognizing an operation input from a user to be accepted thereafter as an operation input in the reduction mode. Thus, in the portable terminal 1 of this embodiment, the pinch-in operation is a switching operation to the reduction mode.

  Thereafter, as shown in FIG. 2B, the user holds the portable terminal 1 in the left hand, touches the thumb on the display portion 105D, and performs a drag operation from the upper side to the lower side as indicated by the dotted down arrow. Or a drag operation from the lower side to the upper side as indicated by the dotted arrow. In this way, the fact that the drag operation in either the up or down direction has been performed is determined based on the change in coordinate information indicating the contact position of the indicator sequentially supplied from the touch pad 105P in the gesture determination unit 110. The amount of operation (the amount of movement of the indicator) can also be grasped. The gesture determination unit 110 notifies the display control unit 112 of the direction in which the drag operation has been performed (the drag operation has been performed and its direction) and the operation amount.

  In this embodiment, the display control unit 112 assumes that the display image immediately after switching to the reduction mode is in a state without reduction. In this case, if a drag operation from the upper side to the lower side is performed, the display control unit 112 gradually increases the reduction ratio of the display image according to the operation amount of the user's drag operation (movement amount of the indicator). To go. Here, gradually increasing the reduction ratio means, for example, increasing the reduction ratio by 10%, displaying the display image at 90% of the original image size, increasing the reduction ratio by 20%, The size of the display image per unit display area is gradually reduced with respect to the original image without reduction (100% image), such as 80% of the size of the original image. means.

  In this way, it is assumed that after the size per unit display area of the display image is gradually reduced, a drag operation is performed from the lower side to the upper side. In this case, the display control unit 112 gradually returns the reduction ratio of the display image according to the operation amount (movement amount) of the user's drag operation. As a result, the display control unit 112 per unit display area of the display image according to the operation amount of the user's drag operation within the range from the state without reduction to the size of 30% of the size of the image without reduction, for example. The size of can be gradually changed. Therefore, the display range of the display image displayed on the display unit 105D can be expanded by reducing the size of the display image per unit display area.

  Here, the reason why the lower limit of reduction is set to 30% of the size of the original image is that it may not be possible to display an image having a size that can be used even if it is reduced further than that. It is. Of course, it is possible to reduce the size to 1% of the size of the original image, or theoretically, it can be made smaller. However, the reduction ratio of the display image is set within a practical range.

  After reducing the display image to the target size using the reduction mode, the user performs a predetermined end operation to exit the reduction mode and maintain normal display while maintaining the reduction ratio of the display image. You can return to mode. After returning to the normal display mode, the display control unit 112 can delete the mode display “reduction” and use the display image with the adjusted reduction ratio by scrolling the display image.

  Note that the predetermined end operation for ending the reduction mode is a drag operation in the left-right direction (direction orthogonal to the up-down direction) on the display unit 105D, a double-tap operation, or the selection of the “end” button displayed on the display unit 105D. Such as operation. The gesture determination unit 110 determines that the operation has been performed and notifies the mode switching unit 111 of the determination. The mode switching unit 111 that has received the notification that the predetermined end operation has been performed controls the display control unit 112 to shift the display processing mode of the own device from the reduced mode to the normal display mode.

  Next, a case where a pinch-out operation is performed on the display unit 105D will be described with reference to FIG. As shown in FIG. 3A, two fingers (for example, thumb and index finger) of the user are brought into contact with the display unit 105D, and the two arrows are displayed as indicated by a large arrow while maintaining the contact state. Assume that a pinch-out operation is performed in which the finger is moved away. In this case, as described above, the gesture determination unit 110 pinches out on the display unit 105D based on the change in the coordinate information indicating the contact positions of the two fingers sequentially provided from the touch pad 105P. It is determined that an operation has been performed, and this is notified to the mode switching unit 111.

  When the mode switching unit 111 receives a notification from the gesture determination unit 110 that a pinch-out operation has been performed, the mode switching unit 111 forms a control signal for switching to the enlargement mode, and supplies this to the display control unit 112. The display control unit 112 first forms a mode display H1 indicating that the own device has been switched to the enlargement mode, and displays this on the upper right end of the display unit 105D as shown in FIG. 3B. Thereafter, the display control unit 112 performs a display control process by recognizing an operation input from a user to be accepted thereafter as an operation input in the enlargement mode. Thus, in the portable terminal 1 of this embodiment, the pinch-out operation is a switching operation to the enlargement mode.

  Thereafter, as shown in FIG. 3B, the user holds the portable terminal 1 in the left hand, touches the thumb on the display unit 105D, and performs a drag operation from the upper side to the lower side as indicated by the dotted down arrow. Or a drag operation from the lower side to the upper side as indicated by the dotted arrow. In this way, the fact that the drag operation in either the upper or lower direction has been performed is a change in the coordinate information indicating the contact position of the indicator (finger) sequentially supplied from the touch pad 105P in the gesture determination unit 110. The amount of operation (the amount of movement of the indicator) can also be grasped. The gesture determination unit 110 notifies the display control unit 112 of the direction in which the drag operation has been performed (the drag operation has been performed and its direction) and the operation amount.

  In this embodiment, the display control unit 112 assumes that the display image immediately after switching to the enlargement mode is not enlarged. In this case, if a drag operation is performed from the lower side to the upper side, the display control unit 112 gradually increases the enlargement ratio of the display image in accordance with the operation amount (movement amount) of the user's drag operation. Go. Here, gradually increasing the enlargement ratio means, for example, increasing the enlargement ratio by 10% and displaying the display image at a size of 110% of the original image, increasing the enlargement ratio by 20%, The display image size per unit display area is gradually reduced with respect to the original image (100% image) without enlargement, such as displaying at a size of 120% of the original image. means.

  In this way, it is assumed that after the size per unit display area of the display image is gradually increased, a drag operation is performed from the upper side to the lower side. In this case, the display control unit 112 gradually returns the enlargement ratio of the display image according to the operation amount (movement amount) of the user's drag operation. Thereby, for example, the display control unit 112 responds to the operation amount of the user's drag operation within a range where the enlargement rate is 200% (twice the size of the original image without enlargement) from the state without enlargement, for example. Can be changed gradually. Therefore, the display range of the display image displayed on the display unit 105D can be narrowed by increasing the size of the display image displayed per unit display area of the display image.

  After enlarging the display image to the target size using the enlargement mode, the user performs a predetermined end operation as in the reduction mode, while maintaining the enlargement ratio of the display image. It is possible to end the enlargement mode and return to the normal display mode. When the display returns to the normal display mode, the display control unit 112 can delete the mode display of “enlargement” and use it by scrolling the display image with the adjusted enlargement ratio.

  Next, a case where a rotation operation is performed on the display unit 105D will be described with reference to FIG. As shown in FIG. 4A, the user's two fingers (eg, thumb and index finger) are brought into contact with the display unit 105D, and the display unit 105D Suppose that a rotation operation for rotating to the right or left is performed. The rotation operation includes the operation of fixing the contact position of one finger and rotating the other finger and the center of the straight line connecting the contact positions of the two fingers or the vicinity thereof. Any operation of rotating a finger of a book may be used. As described above, the gesture determination unit 110 performs a rotation operation on the display unit 105D based on a change in coordinate information indicating each contact position of two fingers sequentially provided from the touch pad 105P. The mode switching unit 111 is notified of this.

  When the mode switching unit 111 receives a notification that the rotation operation has been performed from the gesture determination unit 110, the mode switching unit 111 forms a rotation mode switching control signal and supplies it to the display control unit 112. The display control unit 112 first forms a mode display H1 indicating that the own device has been switched to the rotation mode, and displays this on the upper right end of the display unit 105D as shown in FIG. 4B. Thereafter, the display control unit 112 performs a display control process by recognizing an operation input from a user that is subsequently received as an operation input in the rotation mode. Thus, in the portable terminal 1 of this embodiment, the rotation operation is an operation for switching to the rotation mode.

  Thereafter, as shown in FIG. 4B, the user holds the portable terminal 1 in the left hand, touches the thumb on the display portion 105D, and performs a drag operation from the upper side to the lower side as indicated by the dotted line down arrow. Or a drag operation from the lower side to the upper side as indicated by the dotted arrow. In this way, the fact that the drag operation in either the upper or lower direction has been performed is a change in the coordinate information indicating the contact position of the indicator (finger) sequentially supplied from the touch pad 105P in the gesture determination unit 110. The amount of operation (the amount of movement of the indicator) can also be grasped. The gesture determination unit 110 notifies the display control unit 112 of the direction in which the drag operation has been performed (the drag operation has been performed and its direction) and the operation amount.

  In this embodiment, after switching to the rotation mode, the drag operation from the lower side to the upper side as indicated by the dotted line arrow indicates the right rotation of the display image, and the operation amount of the drag operation is the rotation amount. The amount (rotation angle) is indicated. Further, the drag operation from the upper side to the lower side as indicated by the dotted line down arrow indicates the left rotation of the display image, and the operation amount of the drag operation indicates the rotation amount (rotation angle).

  For this reason, the display control unit 112 rotates and displays the display image according to the direction of the drag operation performed and the operation amount. In this case, the display image is rotated with reference to the center position of the display image. For example, when the map is a display image, so-called annotation (annotation) displays such as place names and facility names are not rotated. So that it won't be difficult to read.

  Accordingly, the display control unit 112 can rotate the display image in the right direction within a range of 0 degrees to 360 degrees by a drag operation from the lower direction to the upper direction. Further, the display control unit 112 can rotate the display image in the left direction in the range of 0 degrees to 360 degrees by a drag operation from the upper direction to the lower direction. Of course, it is also possible to limit the rotation angle. For example, the display image is rotated 0 ° to 180 ° in the right direction by a drag operation from the lower direction to the upper direction, or the display image is moved 0 ° in the left direction by the drag operation from the upper direction to the lower direction. Can be rotated within a range of 180 degrees. As a result, the user can rotate the display image so that the target display state is obtained.

  After using the rotation mode and rotating the display image so as to be in a target state (orientation), the user performs a predetermined end operation as in the reduction mode described above. Thereby, the display control unit 112 can end the rotation mode and return to the normal display mode while maintaining the state (orientation) of the display image. When the display mode returns to the normal display mode, the display control unit 112 can delete the mode display “rotation” and use the rotated display image by scrolling the display image.

  In each processing mode described with reference to FIGS. 2 to 4, the vertical drag operation on the display unit 105 </ b> D is used as an operation for instructing a reduction ratio, an enlargement ratio, a rotation direction, and a rotation amount. I did it. However, it is not limited to this. For example, after switching to the reduction mode, the tap operation to the lower half area of the display unit 105D is used as an instruction input for increasing the reduction rate, and the tap operation to the upper half area of the display unit 105D is reduced. It can be used as an instruction input for returning the rate.

  Similarly, after switching to the enlargement mode, the tap operation to the upper half area of the display unit 105D is used as an instruction input for increasing the enlargement ratio, and the tap operation to the lower half area of the display unit 105D is performed. It can be used as an instruction input for returning the enlargement ratio. In addition, after switching to the rotation mode, the tap operation to the upper half area of the display unit 105D is used as an instruction to input the rotation in the right direction and the rotation amount, and the tap operation to the lower half area of the display unit 105D is performed. It is possible to input a rotation in the left direction and an instruction for the amount of rotation.

  In addition, a long press operation on the upper half area of the display unit 105D can be used to input an instruction to increase the reduction ratio or the enlargement ratio, or to input an instruction to rotate right and the amount of rotation. In addition, a long press operation on the upper half area of the display unit 105D can be used to input an instruction for returning the reduction ratio or the enlargement ratio, or can be used as an instruction input for the left rotation and the rotation amount. That is, various operation inputs other than finger gestures for switching the display processing mode are used as an instruction input for adjusting a reduction ratio or an enlargement ratio, an instruction for a rotation direction, and an instruction input for adjusting a rotation amount. Can do.

[Summary of display processing mode switching and adjustment processing]
Next, the display processing mode switching processing performed in the portable terminal 1 of this embodiment and the adjustment processing related to the display after switching to each mode will be summarized with reference to a flowchart. FIG. 5 is a flowchart for explaining a typical example of the operation input receiving process performed when any display image is displayed on the display unit 105D in the mobile terminal 1 of this embodiment.

  In the mobile terminal 1 of this embodiment, when any image is displayed on the display unit 105D, the control unit 102 executes, for example, an operation input receiving process shown in the flowchart of FIG. Then, the control unit 102 starts accepting operation input from the user through the touch panel 106 including the display unit 105D and the touch pad 105P (step S101). The touch panel 106 detects a contact position by an indicator such as a user's finger at every predetermined timing, and sequentially supplies coordinate information indicating the contact position to the gesture determination unit 110.

  The gesture determination unit 110 determines what operation has been performed on the touch panel 106 by the user under the control of the control unit 102 (steps S102 to S105). First, the gesture determination unit 110 determines whether or not two or more positions on the display unit 105D are instructed by the indicator (step S102). When it is determined in the determination process of step S102 that two or more positions are instructed, the gesture determination unit 110 determines whether a pinch-in operation has been performed (step S103).

  If it is determined in step S103 that the pinch-in operation has not been performed, the gesture determination unit 110 determines whether a pinch-out operation has been performed (step S104). If it is determined in step S104 that the pinch-out operation has not been performed, the gesture determination unit 110 determines whether a rotate operation has been performed (step S105). If it is determined in the determination process in step S105 that the rotation operation has not been performed, a valid operation input has not been received, and the process from step S102 is repeated.

  When it is determined in the determination process of step S102 that two or more positions are not instructed, the gesture determination unit 110 notifies the display control unit 112 of what operation has been performed, and the accepted operation is performed. A corresponding process is executed (step S106). In this case, the operation input accepted through the touch panel 106 is an operation such as a drag operation or a tap operation performed by one indicator. For this reason, in step S106, a process according to an operation input from the user, such as a display image scroll process, is performed.

  Further, it is assumed that it is determined in the determination process in step S103 that a pinch-in operation has been performed. In this case, the gesture determination unit 110 notifies the mode switching unit 111 that the pinch-in operation has been performed, and the mode switching unit 111 controls the display control unit 112 to execute the reduction mode process (step S107).

  Also, assume that it is determined in the determination process in step S104 that a pinch-out operation has been performed. In this case, the gesture determination unit 110 notifies the mode switching unit 111 that the pinch-out operation has been performed, and the mode switching unit 111 controls the display control unit 112 to execute an enlargement mode process (step S108). Further, it is assumed that in the determination process in step S105, it is determined that the rotation operation is not performed. In this case, the gesture determination unit 110 notifies the mode switching unit 111 that the rotation operation has been performed, and the mode switching unit 111 controls the display control unit 112 to execute the rotation mode process (step S109).

  After the processing according to the operation in step S106, the reduction mode processing in step S107, the same mode processing in 108, and the rotation mode processing in step S109 are completed, the control unit 102 repeats the processing from step S102. . Thus, in the portable terminal 1, when a display image is displayed on the display unit 105D, the mode is switched to the reduction mode, the enlargement mode, and the rotation mode by the pinch-in operation, the pinch-out operation, and the rotation operation, respectively. The display process according to the mode can be performed.

[Details of reduction mode processing, enlargement mode processing, and rotation mode processing]
Next, details of each of the reduction mode process, the enlargement mode process, and the rotation mode process performed in steps S107 to S108 of the operation input reception process shown in FIG. 5 will be described. Note that the reduction mode process, the enlargement mode process, and the rotation mode process described below are executed by the gesture determination unit 110, the mode switching unit 111, and the display control unit 112 under the control of the control unit 102. It is processing.

[Reduction mode processing]
FIG. 6 is a flowchart for explaining the reduction mode process executed in step S107 of the operation input reception process shown in FIG. Receiving the notification that the pinch-in operation has been performed, the mode switching unit 111 generates a switching signal to the reduction mode and supplies it to the display control unit 112, thereby switching the display control unit 112 to the reduction mode ( Step S201). The display control unit 112 displays the reduction mode display H1 (FIG. 2B) on the display unit 105D (step S202). Thereafter, the gesture determination unit 110 receives an operation input from the user through the touch panel 106 (step S203).

  The gesture determination unit 110 determines whether the operation input received through the touch panel 106 is an end operation for ending the reduction mode (step S204). When it is determined in the determination process of step S204 that the end operation has been accepted, the gesture determination unit 110 switches the display control unit 112 to the normal display mode (step S205), ends the process shown in FIG. It returns to the process of step S107 shown.

  If it is determined in the determination process in step S204 that the end operation has not been accepted, the gesture determination unit 110 determines whether or not the operation is related to adjustment in the reduction mode, such as a vertical drag operation (step S206). If it is determined in the determination process in step S206 that the operation is not an operation related to the adjustment, an effective operation related to the adjustment in the reduction mode is not performed, and thus the processing from step S203 is repeated.

  If it is determined in the determination process in step S206 that an operation related to adjustment of the reduction mode has been performed, the reduction ratio is changed according to the operation, and the display image is displayed again (step S207). Thereafter, the processing from step S203 is repeated. Thus, after the reduction mode is set, as described above, the display image can be displayed again at the target reduction ratio by performing a drag operation in the vertical direction.

  FIG. 7 is a diagram for explaining a display example in the reduction mode when the display image is a map image. 7A is a map image in an initial display state displayed on the display unit 105D before switching to the reduction mode. In this case, after switching to the reduction mode by performing a pinch-in operation, the reduction rate can be adjusted by performing a vertical drag operation indicated by a dotted arrow in FIG. 7A. it can.

  Thereby, when a drag operation from the upper side to the lower side is performed, the reduction rate is increased, and a map image with the reduced reduction rate can be displayed as shown in FIGS. 7B and 7C. In the case of the example shown in FIG. 7, the reduction rate of the map image is gradually increased in the order of FIG. 7 (A) → FIG. 7 (B) → FIG. 7 (C), and the display of the map image itself is small. However, the displayed range can be adjusted to be wide. That is, it is possible to easily display a wider range of map images.

[Enlarged mode processing]
FIG. 8 is a flowchart for explaining the enlargement mode process executed in step S108 of the operation input reception process shown in FIG. Upon receiving the notification that the pinch-out operation has been performed, the mode switching unit 111 forms a switching signal to the enlargement mode and supplies this to the display control unit 112, thereby switching the display control unit 112 to the enlargement mode. (Step S301). The display control unit 112 displays the enlarged mode display H1 (FIG. 3B) on the display unit 105D (step S302). Thereafter, gesture determination unit 110 accepts an operation input from the user through touch panel 106 (step S303).

  The gesture determination unit 110 determines whether the operation input received through the touch panel 106 is an end operation for ending the enlargement mode (step S304). If it is determined in the determination process in step S304 that the end operation has been accepted, the gesture determination unit 110 switches the display control unit 112 to the normal display mode (step S305), ends the process shown in FIG. It returns to the process of step S108 shown.

  If it is determined in the determination process in step S304 that the end operation has not been accepted, the gesture determination unit 110 determines whether or not the operation is related to adjustment in the enlargement mode, such as a vertical drag operation (step S306). When it is determined in the determination process in step S306 that the operation is not related to the adjustment, the effective operation related to the adjustment in the enlargement mode is not performed, and thus the processing from step S303 is repeated.

  If it is determined in the determination process of step S306 that an operation related to adjustment of the enlargement mode has been performed, the enlargement ratio is changed according to the operation, and the display image is displayed again (step S307). Thereafter, the processing from step S303 is repeated. As a result, after the enlargement mode is set, the display image can be displayed again at the target enlargement ratio by performing a drag operation in the vertical direction as described above.

  A display example when the display image is adjusted using the enlargement mode is not particularly shown. However, when the display image is a map image, the lower side is opposite to the display example in the reduction mode shown in FIG. The enlargement ratio of the map image can be increased by the drag operation from the top to the bottom. In this case, when the enlargement ratio is increased, the display of the map image itself is increased, but the display range can be adjusted to be narrowed. That is, it is possible to easily display a map image focusing on the target area.

[Rotation mode processing]
FIG. 9 is a flowchart for explaining the rotation mode process executed in step S109 of the operation input reception process shown in FIG. Receiving the notification that the rotation operation has been performed, the mode switching unit 111 forms a switching signal to the rotation mode and supplies it to the display control unit 112, thereby switching the display control unit 112 to the rotation mode ( Step S401). The display control unit 112 displays the rotation mode display H1 (FIG. 4B) on the display unit 105D (step S402). Thereafter, the gesture determination unit 110 receives an operation input from the user through the touch panel 106 (step S403).

  The gesture determination unit 110 determines whether or not the operation input received through the touch panel 106 is an end operation for ending the enlargement mode (step S404). If it is determined in the determination process of step S404 that the end operation has been accepted, the gesture determination unit 110 switches the display control unit 112 to the normal display mode (step S405), ends the process shown in FIG. The process returns to the process of step S109 shown.

  If it is determined in the determination process in step S404 that the end operation has not been accepted, the gesture determination unit 110 determines whether or not the operation is related to adjustment in the rotation mode, such as a vertical drag operation (step S406). When it is determined in the determination process in step S406 that the operation is not related to the adjustment, an effective operation related to the adjustment of the rotation mode is not performed, and thus the process from step S403 is repeated.

  If it is determined in the determination processing in step S406 that an operation related to the adjustment of the rotation mode has been performed, the display image is rotated in the direction corresponding to the operation by the amount of rotation corresponding to the operation, and the display image is displayed again ( Step S407). Thereafter, the processing from step S403 is repeated. Thus, after the rotation mode is set, as described above, the display image can be rotated and displayed again in the target direction by the drag operation in the vertical direction. it can.

  FIG. 10 is a diagram for explaining a display example in the rotation mode when the display image is a map image. FIG. 10B is a map image in a so-called initial display state displayed on the display unit 105D before switching to the rotation mode. In this case, after the rotation operation is performed, the rotation direction and the rotation amount are adjusted by performing a drag operation in the vertical direction indicated by a dotted arrow in FIG. 10B after switching to the rotation mode. It can be carried out.

  Thus, when a drag operation is performed from the upper side to the lower side, for example, as shown in FIG. 10A, the map is moved by the amount operated leftward from the state shown in FIG. 10B. Images can be rotated and displayed. Further, when a drag operation from the lower side to the upper side is performed, for example, as shown in FIG. 10C, the map image is an amount operated rightward from the state shown in FIG. 10B. Can be rotated for display. In particular, in the case of a map image, it is easy to see according to the orientation of the user without changing the orientation of the mobile terminal 1 itself depending on the direction the user is facing. You can adjust the orientation of the map image to the state.

[Effect of the embodiment]
In the mobile terminal 1 of the above-described embodiment, the display processing mode (reduction mode, enlargement mode, rotation mode) intended by the user can be switched by an easy-to-understand operation such as a pinch-in operation, a pinch-out operation, and a rotate operation. . After switching to each mode, for example, the reduction ratio and the enlargement ratio, the rotation direction, and the rotation amount can be adjusted by a simple operation such as a drag operation in the vertical direction.

  Accordingly, as shown in FIGS. 2 to 4, when the portable terminal 1 is held in the left hand, for example, the user can perform a pinch-in operation, a pinch-out operation, and a rotate operation on the display unit 105 </ b> D with the right hand. Can switch to the intended display processing mode. After switching to the target display processing mode, the adjustment operation can be performed with only the left hand.

  In various conventional terminal devices, a pinch-in operation, a pinch-out operation, and a rotate operation are instructions for performing an execution instruction and an operation amount, such as a reduction instruction and a reduction amount, an enlargement instruction and an enlargement amount, and a rotation instruction and a rotation amount. It was. For this reason, if the reduction amount, enlargement amount, and rotation amount are not the amounts intended by the user, it is necessary to repeat the pinch-in operation, the pinch-out operation, and the rotate operation, with the terminal device on the left hand and the right hand It was necessary to use both hands from start to finish, such as performing gesture operations.

  However, in the case of the portable terminal 1 of this embodiment, as shown in FIGS. 2 to 4, one hand (left hand in FIGS. 2 to 4) is obtained after switching to the target mode by a gesture operation. The adjustment operation can be performed only with the other hand, and the other hand (the right hand in FIGS. 2 to 4) is free, and the display image can be easily adjusted while taking notes. Further, since the adjusting operation can be performed by, for example, a drag operation, a gesture operation such as a pinch-in operation, a pinch-out operation, and a rotate operation is not repeated many times as in the past. Therefore, it is possible to easily and appropriately set the display image to a target state.

[Modifications, etc.]
In the above-described embodiment, the pinch-in operation is the switching operation for the reduction mode and the pinch-out operation is the switching operation for the enlargement mode. However, the present invention is not limited to this. For example, a pinch operation and a pinch out operation are used as a switching operation of a display image enlargement / reduction mode. The enlargement ratio can be changed by a drag operation from the lower side to the upper side, and the reduction ratio can be changed by a drag operation from the upper side to the lower side.

  In the above-described embodiment, the rotation operation is switched to the rotation mode in any direction, but the present invention is not limited to this. The clockwise rotation operation may be a switching operation to the clockwise rotation mode, and the counterclockwise rotation operation may be a switching operation to the counterclockwise rotation mode. After switching to the clockwise rotation mode or the counterclockwise rotation mode, the drag operation from the lower side to the upper side is an operation to increase the rotation amount, and the operation from the upper side to the lower side is increased. The operation is to reduce the amount of rotation. As a result, the rotation direction can be clearly distinguished, the user can rotate the target amount by the target amount, and the rotation amount can be adjusted appropriately.

  As described above, the instruction operation for adjustment after switching to each display processing mode can be performed by various operations such as a drag operation, one or more tap operations, a long press operation, and one or more flick operations. Can be used. In addition, as described above, operations for ending each display processing mode include switching the display processing mode such as a drag operation in the left-right direction, a double-tap operation, an “end” button selection operation, and a drag operation in the diagonal direction. Various operations different from the operation can be set as the predetermined end operation.

  Further, as described above, the present invention is not limited to high-function mobile phone terminals, but includes touch panels such as tablet PCs, personal computers, in-vehicle navigation devices, ticket machines that sell bank ATMs, CDs, tickets, and the like. The present invention can be applied to various electronic devices including In particular, it is suitable for application to a high-function mobile phone terminal, a tablet PC, an in-vehicle navigation device, and the like that require operation with one hand.

[Others]
As can be understood from the description of the above-described embodiment, the function as the contact position detection unit of the image display processing device is realized by the touch pad 105P of the touch panel 106 of the mobile terminal 1, and the function of the determination unit is the gesture determination unit. 110 is realized. In addition, the function of the mode switching unit of the image display processing device is realized by the mode switching unit 111 of the mobile terminal 1, and the functions of the mode display processing unit and the display control unit are realized by the display control unit 112.

  The processing shown in the flowcharts of FIGS. 5, 6, 8, and 9 is the one to which an embodiment of the image display processing method of the present invention is applied. A program for performing the processing shown in the flowcharts of FIGS. 5, 6, 8, and 9 is one in which an embodiment of the image display processing program of the present invention is applied.

  The functions of the gesture determination unit 110, the mode switching unit 111, and the display control unit 112 illustrated in FIG. 1 can also be realized as functions of the control unit 102 by a program executed by the control unit 102.

  DESCRIPTION OF SYMBOLS 1 ... Portable terminal, 101 ... Wireless communication part, 101A ... Transmission / reception antenna, 102 ... Control part, 103 ... Memory | storage device, 104 ... GPS part, 104A ... GPS antenna, 105D ... Display part, 105P ... Touchpad, 106 ... Touch panel, 110: Gesture discriminating unit, 111: Mode switching unit, 112 ... Display control unit

Claims (7)

Contact position detecting means for detecting the one or more contact positions by an indicator that is in contact with one or more positions on a display unit on which an image is displayed;
The detection result from the contact position detecting means is sequentially received, and based on the change in the contact position of each indicator that is in contact with at least two points on the display unit, the indicator is passed through the indicator on the display unit. A discriminating means for discriminating an operation performed on the device;
Mode switching means for switching to a processing mode corresponding to the operation determined by the determination means;
Processing mode display processing means for performing display on the display unit to notify the processing mode switched by the mode switching means;
After the processing mode is switched by the mode switching unit, the display unit according to the operation performed by the user determined based on the detection result from the contact position detection unit and the switched processing mode An image display processing device comprising: display control means for changing a display mode of an image displayed on the display. The image display processing device according to claim 1,
The discriminating means is configured such that at least one indicator moves on the display unit while each indicator that is in contact with at least two points on the display unit maintains contact on the display unit. When the distance between the contact positions of the respective indicators that are in contact with at least two points on the display unit is shortened, it is determined that the operation is an instruction to reduce the image;
The mode switching means switches to an image reduction processing mode when the operation determined by the determination means is an operation for instructing image reduction.
The display control means sequentially receives the detection results from the contact position detection means after the mode reduction means is switched to the image reduction processing mode, and uses the indicator for the display section. An image display processing device, wherein when a tap operation or a drag operation is performed, an image displayed on the display unit is displayed with a reduced reduction rate. The image display processing device according to claim 1, wherein:
The discriminating means is configured such that at least one indicator moves on the display unit while each indicator that is in contact with at least two points on the display unit maintains contact on the display unit. When the distance between the contact positions of the respective indicators that are in contact with at least two points on the display unit is increased, it is determined that the operation is an instruction to instruct the enlargement of the image,
The mode switching means switches to an image enlargement processing mode when the operation determined by the determination means is an operation for instructing image enlargement,
After the display control means is switched to the image enlargement processing mode by the mode switching means, the display control means sequentially receives the detection results from the contact position detection means, and the display unit uses the indicator. An image display processing device that displays an image displayed on a display unit while changing an enlargement ratio when a tap operation or a drag operation is performed. An image display processing device according to any one of claims 1, 2 and 3,
The discriminating means moves one or both of the indicators so as to draw a circle while each indicator contacted with at least two points on the display unit maintains contact with the display unit. In the case, it is determined that the operation is an instruction to rotate the image,
The mode switching means switches to an image rotation processing mode when the operation determined by the determination means is an operation to instruct rotation of an image,
After the display control means is switched to the image rotation processing mode by the mode switching means, the display control means sequentially receives the detection results from the contact position detection means, and the display unit uses the indicator. An image display processing device characterized in that when a tap operation or a drag operation is performed, an image displayed on the display unit is displayed by controlling a rotation direction and a rotation angle according to the operation. An image display processing device according to any one of claims 1, 2, 3, or 4,
The image displayed on the display unit is a map image. A contact position detection step of detecting the one or more contact positions by an indicator that is brought into contact with one or more positions on the display unit on which an image is displayed, through the contact position detection means;
The detection result in the contact position detection step is sequentially received, and the display unit displays the display through the indicator based on a change in the contact position of each indicator that is in contact with at least two points on the display unit. A discriminating step for discriminating an operation performed on the department;
A mode switching step in which the mode switching means switches to a processing mode according to the operation determined in the determination step;
A processing mode display processing step in which processing mode display processing means displays a display notifying of the processing mode switched in the mode switching step;
After the processing mode is switched in the mode switching step, the display processing means performs the display according to the operation performed by the user determined based on the detection result in the contact position detection step and the switched processing mode. A display control step of changing a display mode of an image displayed on the unit. A computer mounted on a display processing apparatus having a display unit,
A contact position detecting step of detecting the one or more contact positions by an indicator that has been brought into contact with one or more positions on the display unit on which an image is displayed through the contact position detecting means;
The detection result in the contact position detection step is sequentially received, and the display unit displays the display through the indicator based on the change in the contact position of each indicator that is in contact with at least two points on the display unit. A determination step for determining an operation performed on the unit;
A mode switching step in which the mode switching means switches to a processing mode according to the operation determined in the determination step;
A processing mode display processing step in which processing mode display processing means displays a display notifying of the processing mode switched in the mode switching step;
In accordance with the operation performed by the user determined based on the detection result in the contact position detection step after switching the processing mode in the mode switching step and the switched processing mode, the display processing means And a display control step for changing a display mode of an image displayed on the display unit.

Images