JP6034140B2 - Display device, display control method, and program - Google Patents

Description

  The present invention relates to a technique for displaying an image that functions as a user interface in a display device.

  Some display devices that can be operated by a user while holding the hand display an image on the front surface, and have a function of receiving a user operation input from the back surface. Patent Document 1 discloses that a user's contact operation on the back surface of a display device such as a digital photo frame is detected, and various processes (such as fast-forwarding, rewinding, and pausing a slide show) are performed on an image displayed on the front surface. Has been.

JP 2010-108071 A

  By the way, when a user uses a display device, it cannot always be operated using both hands. For example, if a user holds a luggage with a hand that does not have a display device or is holding a strap of a train or a bus, the display device must be operated only with the hand holding the display device. Thus, even when the user has to operate the display device with one hand, it is desirable that the operation burden on the user be as small as possible.

  Further, according to the mechanism described in Patent Document 1, various processes such as fast forward, rewind, and pause can be individually instructed by changing the contact method when the user touches the back surface of the display device with a finger. it can. However, when there are many types of processes, the user cannot quickly instruct a desired process unless he / she remembers the correspondence between the contact method and the process. Therefore, it can be said that the burden regarding a user's operation is large.

  Accordingly, an object of the present invention is to reduce the burden on the user when instructing the display device to perform various processes.

In order to solve the above-described problems, the present invention provides a display surface for displaying an image, an operation surface provided on the opposite side of the display surface, and an operation for detecting an operation performed by a user on the operation surface. When an operation is detected by the detection unit and the operation detection unit, the user instructs the processing performed by the user apparatus to a position on the display surface corresponding to the position on the operation surface where the operation is performed. Display control means for displaying a user interface image for the first user interface image for processing the content displayed on the display surface in response to an operation on the operation surface, or the display Hisage a display device characterized by comprising a display control means for displaying either the second user interface image without the content being displayed on the surface processed to the display surface To.

Before Symbol display control means, even when either the first user interface image or the second interface image in accordance with the position on the operation has been performed the operation surface so as to be displayed on the display surface Good.

Before Symbol display control means displays any one of the first user interface image or the second interface image in accordance with the magnitude of the force at which the user performs an operation on the operation surface on the display surface You may make it make it.

Before Symbol display control means so as to display either the first user interface image or the second interface image in accordance with the operation amount when the user performs an operation on the operation surface on the display surface It may be.

The present invention also includes a step of detecting an operation performed by a user on an operation surface provided on the side opposite to a display surface for displaying an image, and when an operation on the operation surface is detected, the operation is performed. Displaying a user interface image for instructing processing performed by the user at a position on the display surface corresponding to the performed position on the operation surface, the operation on the operation surface Depending on the above, either the first user interface image whose processing target is the content displayed on the display surface or the second user interface image which does not target the content displayed on the display surface the provides a display control method characterized by comprising the step of displaying on the display surface.

Further, according to the present invention, a step of detecting an operation performed by a user on an operation surface provided on a side opposite to a display surface for displaying an image on a computer, and an operation on the operation surface are detected. Displaying a user interface image for instructing processing performed by the user at a position on the display surface corresponding to a position on the operation surface where the operation is performed , the operation surface The first user interface image whose content is displayed on the display surface in response to an operation on the display screen, or the second user interface image which does not target the content displayed on the display surface A program for executing any one of the above is displayed on the display surface .

  According to the present invention, it is possible to reduce the burden on the user when instructing the display device to perform various processes.

The block diagram which shows the hardware constitutions of a display apparatus. FIG. The figure which shows a mode when a user hold | grips a display apparatus. The block diagram which shows the function structure of a display apparatus. The flowchart which shows the procedure of a process of a control part. The figure explaining a mode that a user interface image changes. The figure explaining the example of a display of a user interface image. The figure explaining the example of a display of a user interface image, and a user's operation. The figure explaining the example of a display of a user interface image, and a user's operation.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[Embodiment]
FIG. 1 is a block diagram showing a hardware configuration of a display device 10 according to an embodiment of the present invention. The display device 10 is a device that can be operated while being held by a user and has at least a function of displaying an image, such as a mobile phone, a smartphone, a wireless local area network (LAN) terminal, or a tablet terminal. As shown in FIG. 1, the display device 10 includes a control unit 11, an operation unit 12, a communication unit 13, a display unit 14, and a storage unit 15. The control unit 11 is a microcomputer having a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). The CPU controls each unit of the display device 10 by reading a program stored in the ROM or the storage unit 15 into the RAM and executing the program. The operation unit 12 includes a back surface touch pad 121, a front surface sensor 122, and a physical key (not shown), and receives an operation by a user. The communication unit 13 has a wireless communication circuit and an antenna, and is an interface for communicating with a network (not shown). The display unit 14 is a liquid crystal display, for example, and has a rectangular display surface 141 in which a plurality of pixels are arranged. The storage unit 15 is a storage device such as an EEPROM (Electronically Erasable and Programmable ROM) or a flash memory, and stores programs executed by the control unit 11 and contents displayed on the display unit 14. The content displayed on the display unit 14 is, for example, an electronic book, and is stored in the storage unit 15 by a method such as downloading to the display device 10 by the communication unit 13.

  2A is an external view when the surface of the display device 10 is viewed in plan, and FIG. 2B is an external view when the back surface of the display device 10 is viewed in plan. When the user holds the display device 10, the surface facing the user is the front surface, and the opposite surface is the back surface. The display device 10 has a size that can be held by a user in one hand, and has a plate shape (in other words, a thin rectangular parallelepiped shape) as a whole. The back surface touch pad 121 functions as an operation surface provided on the back surface of the display device 10. The back surface touch pad 121 detects a contact operation performed using a user's finger or the like using, for example, a contact sensor. When the back surface touch pad 121 detects the position indicated by the contact operation, the back surface touch pad 121 supplies position information (coordinate information on the back surface touch pad 121) indicating the detected position to the control unit 11. The back surface touch pad 121 is, for example, a capacitance type sensor, but the detection method of the sensor is not particularly limited. The operation detected by the back surface touch pad 121 is hereinafter referred to as “back surface operation”.

  The surface sensor 122 is provided so as to overlap the display surface 141 and functions as an operation surface provided on the surface of the display device 10. The surface sensor 122 is made of, for example, a transparent material so as not to prevent the user from visually recognizing the image displayed on the display surface 141. The surface sensor 122 is a sensor that detects a state in which a user's finger or the like is in contact with the display surface 141. The surface sensor 122 is, for example, a capacitance type sensor, but the detection method of the sensor is not particularly limited. When the surface sensor 122 detects a position indicated by a user's finger or the like, the surface sensor 122 supplies position information (coordinate information in the surface sensor 122) indicating the detected position to the control unit 11. The operation detected by the surface sensor 122 is hereinafter referred to as “surface operation”.

  In FIG. 2, the display surface 141 is larger than the back surface touch pad 121, but this is only an example, and the display surface 141 and the back surface touch pad 121 may be approximately the same size. However, when viewed from a direction perpendicular to the display surface 141, the positional relationship is such that the display surface 141 and the back surface touch pad 121 overlap.

  3A shows a state when the user holds the display device 10 with the right hand and browses an image displayed on the display surface 141, and FIG. 3B shows a state where the user holds the display device 10 with the right hand. A state when the back surface operation is performed while gripping is shown. As shown in FIG. 3A, when viewing an image, the user holds the display device 10 so as to wrap the image from the back side with the right hand. When the user performs the back surface operation using the right hand, as shown in FIG. 3B, for example, in the back surface touch pad 121 of the display device 10, the surface of the back surface touch pad 121 is lightened within the reach of the index finger f1. Press (touch). In response to this, the user instructs the processing performed by the display device 10 at a position on the display surface 141 corresponding to the operation position (the position of the fingertip of the index finger f1) on the back surface touchpad 121 where the operation is performed. The user interface image UI is displayed.

  Here, “the position on the display surface 141 corresponding to the operation position on the back surface touch pad 121” is a correspondence that the position on the display surface 141 is uniquely determined if the operation position on the back surface touch pad 121 is determined. A position that is in a relationship. Typically, the position on the display surface 141 is a position on the display surface 141 that overlaps with the operation position on the back touch pad 121 when viewed from a direction perpendicular to the display surface 141. In this case, as shown in FIG. 3B, when viewed from the user, it is felt that the user interface image UI is displayed at the position of the fingertip of the index finger f1.

  The user interface image is an image for exchanging information between the display device 10 and the user. For example, an object such as an icon or a soft button associated with the process is indicated by an indicator such as a user's finger. A graphical user interface (GUI) for In the example of FIG. 3B, the user interface image UI creates an e-mail and an icon i1 for instructing a process for storing data, an icon i2 for instructing a process for newly displaying data. It includes an icon i3 for instructing processing. The user touches an icon included in the user interface image UI displayed on the display surface 141 with the thumb f2 or the like to instruct the display device 10 to perform a desired process, and the control unit 11 of the display device 10 performs the process. Execute. Since the user interface image UI is configured to be able to interact with the user, the user can operate according to the guidance of the user interface image UI even when there are many types of processing as options for the user. Since it is only necessary, the operation burden is small.

  Further, as described above, since the user interface image UI is displayed at a position within the reach of the user's index finger f1, the thumb f2 can easily reach the position on the display surface 141, and an operation for selecting an icon is performed. Will also be easier. If the user touches the back side of the background area or the like where content is not displayed with the index finger f1 in the state of FIG. 3A, the user interface image UI is displayed in an area that does not interfere with visual recognition of the content. Is possible. That is, the user can display the user interface image UI at a position that he / she wishes to display for various reasons. In the case where the user holds the display device 10 with his / her left hand, “right hand” may be read as “left hand” in the description related to FIG. 3.

  FIG. 4 is a block diagram illustrating a functional configuration of the display device 10. As illustrated in FIG. 4, the control unit 11 of the display device 10 realizes functions as an operation detection unit 111 and a display control unit 112. The operation detection unit 111 detects an operation performed on the operation unit 12 by the user. The operation detection unit 111 acquires position information from the back surface touch pad 121 and detects a back surface operation according to the acquired position information. In addition, the operation detection unit 111 acquires position information from the surface sensor 122 and detects a surface operation according to the acquired position information.

  The display control unit 112 controls the display unit 14 to display an image on the display surface 141. The display control unit 112 performs display control according to the surface operation detected by the operation detection unit 111. For example, when an object such as an icon displayed on the display surface 141 is selected, the display control unit 112 displays an image corresponding to the selected object. The display control unit 112 performs display control according to the back surface operation detected by the operation detection unit 111. For example, the display control unit 112 displays the above-described interface image UI at a position on the display surface 141 corresponding to the position on the back surface touch pad 121 where the back surface operation is performed.

  Next, the operation of this embodiment will be described. FIG. 5 is a flowchart illustrating a procedure of processing related to display of the user interface image by the control unit 11. For example, when the control unit 11 detects a surface operation for selecting content to be browsed by the user while being held by the user in the manner illustrated in FIG. 3A, the control unit 11 displays the selected content on the display surface 141. . The control unit 11 determines whether a back surface operation is detected by the back surface touch pad 121 during display of the content (step S11).

  When detecting the back surface operation (step S11; YES), the control unit 11 converts the coordinate information on the back surface touch pad 121 into the coordinate information on the surface sensor 122, thereby displaying the display surface corresponding to the operation position of the back surface operation. The position on 141 is specified (step S12). And the control part 11 displays the user interface image UI in the position on the specified display surface 141 (step S13).

  Thereby, the user interface image UI is displayed on the display surface 141 as shown in FIG. FIG. 6A is an enlarged view of the user interface image UI. In addition to the icons i1 to i3 described above, the user interface image UI includes soft buttons S11 and S12 for updating the user interface image. When the control unit 11 determines that any of the icons i1 to i3 is selected by the user and the processing is instructed (step S14; YES), the control unit 11 executes the processing associated with the selected icon ( Step S15).

  In the present embodiment, five or more icons are virtually arranged in a ring shape, and when the soft buttons i11 and i12 are selected, the position of the icon is one as if the ring is rotated. The user interface image UI is displayed in a shifted state. For example, when the soft button S11 pointing to the right direction is selected, the control unit 11 determines that the user interface image UI has been sent (step S16; YES) and is virtually arranged in a ring shape. The user interface image UI is updated and displayed so that the position of the selected icon is moved to the left by one (step S17).

  As a result, a user interface image UI as shown in FIG. 6B is displayed on the display surface 141. This user interface image UI includes an icon i4 for instructing image enlargement processing in addition to the icons i2 and i3 described above, but does not include the icon i1. On the other hand, when the soft button S10 pointing to the left direction is selected, the control unit 11 determines that the user interface image feed operation has been performed, and the user moves the icon position to the right by one. The interface image UI is updated and displayed.

  When the user wants to delete the user interface image UI from the display surface 141, the user may touch the position on the back touch pad 121 corresponding to the display position of the user interface image UI. When the control unit 11 displays the user interface image UI on the display surface 141 and then detects that the position on the back touch pad 121 corresponding to the display position of the user interface image UI is operated (step S18; YES) ), The display of the user interface image is stopped and only the content is displayed (step S19).

  Further, when the user wants to change the display position of the user interface image UI, a position other than the position on the back touch pad 121 corresponding to the display position of the user interface image UI (that is, a new display of the user interface image UI). The position on the back surface touchpad 121 corresponding to the position may be touched. When the control unit 11 displays the user interface image UI on the display surface 141, the control unit 11 detects that a position other than the position on the back surface touchpad 121 corresponding to the display position of the user interface image UI is operated (Step S11). S20; YES), the user interface image UI that has been displayed so far is deleted, and a new user interface image UI is displayed at a position on the display surface 141 corresponding to the operation position (step S21). That is, it seems to the user that the user interface image UI has moved.

  As described above, according to the present embodiment, the user performs the back surface operation to display the user interface image UI at a desired position on the display surface 141, and the front surface operation for the user interface image UI is performed. Thus, it is possible to instruct the display device 10 to perform a desired process. Therefore, even when there are many types of processing that are options for the user, the user only has to operate according to the guidance of the user interface image UI. For example, the correspondence between the type of processing and the operation method is stored. There is no need.

  Since the user interface image UI is displayed at a position within the reach of the fingertip of the user's index finger f1, the thumb f2 can easily reach the position on the display surface 141, and an icon for instructing processing is selected. Operation is also easy.

  In addition, if the user touches the back surface portion such as a background area where content is not displayed with the index finger f1, the user interface image UI can be displayed in an area that does not interfere with visual recognition of the content.

  In addition, when designating the display position of the user interface image UI, it is also possible to designate by the front surface operation instead of the back surface operation. However, if such a mechanism is used, the content displayed on the display surface 141 is blocked by the thumb when performing a surface operation, and thus the visual recognition of the content is hindered at this time. On the other hand, according to the present embodiment, the visual recognition of the content is not hindered in the process until the user interface image UI is displayed.

[Modification]
The present invention can be implemented in a form different from the above-described embodiment. The present invention can also be implemented in the following forms, for example. Further, the following modifications may be combined as appropriate.
(Modification 1)
In the embodiment, the user interface image UI having a different content is displayed by selecting a soft button indicating a feed operation for moving the position of the icon. However, the user interface image UI is displayed as follows. It may be.

  For example, the control unit 11 may display different user interface images on the display surface 141 according to the position on the back touch pad 121 where the operation is performed. As shown in FIG. 7A, for example, if a predetermined central area of the back surface touchpad 121 is operated, the control unit 11 displays the first user interface image with the displayed content as a processing target. UI1 (for example, a user interface image for instructing processing such as content storage, copying, and enlargement) is displayed. On the other hand, as shown in FIG. 7B, if the predetermined end region of the back surface touchpad 121 is operated, the control unit 11 causes the second user not to process the displayed content. An interface image UI2 (for example, a user interface image for instructing processing such as activation of other software such as a mailer or voice recognition software) is displayed.

Further, the control unit 11 may display different user interface images on the display surface according to the magnitude of the force when the user performs an operation on the back surface touch pad 121. For example,
The back surface touch pad 121 has a function of detecting a pressing force when pressed by a user's finger. And the control part 11 detects the pressing force in back surface operation, and as shown to FIG. 8 (A), if the pressing force is more than a threshold value, it will be the 1st user who processed the displayed content as a process target The interface image UI1 is displayed. As shown in FIG. 8B, if the pressing force is less than the threshold value, the second user interphase image UI2 that does not target the displayed content is displayed.

Further, the control unit 11 may display different user interface images on the display surface according to the operation amount when the user performs an operation on the back surface touch pad 121. For example, the control unit 11 displays different user interface images according to the number of times (operation amount) that the operation of pressing the back surface touch pad 121 is continuously performed by the back surface operation. Specifically, if the pressing operation is performed once, the control unit 11 displays the first user interface image UI1 for processing the displayed content, and the pressing operation is continuously performed twice within a predetermined period. If there is, the second user interphase image UI2 that does not target the displayed content is displayed.
In the above-described embodiments and modifications, specific numerical values and modes that define the operation method of the back surface operation detected by the display device 10 are merely examples, and may be appropriately changed to other numerical values and modes.

(Modification 2)
The display device 10 may have different operation modes, that is, a two-hand operation mode and a one-hand operation mode, and may display a user interface image according to the back surface operation only in the one-hand operation mode. In this case, display control in the two-hand operation mode may be performed in any manner, for example, display control may be performed only by surface operation. Regarding the determination of the operation mode, the display device 10 may specify whether the operation mode manually set by the user is the two-handed operation mode or the one-handed operation mode, or detects the front surface operation or the back surface operation. The result may be analyzed to identify the operation mode automatically.

(Modification 3)
In the embodiment, the control unit 11 displays the user interface image only in accordance with the back surface operation. However, the control unit 11 detects the back surface operation (first operation) and the front surface operation (second operation). You may make it display a user interface image on the display surface 141 according to 1st operation) and surface operation (2nd operation). As shown in FIG. 9A, for example, when the control unit 11 detects a back surface operation (first operation) and then has a front surface operation (second operation) that touches a predetermined position on the display surface 141. The user interface image UI may be displayed on the display surface. Further, as shown in FIG. 9B, when the control unit 11 detects the back surface operation (first operation) after the front surface operation (second operation) touching a predetermined position on the display surface 141, The user interface image UI may be displayed on the display surface. In addition, as illustrated in FIG. 9C, the control unit 11 performs the back surface operation (first operation) during the period during which the front surface operation (second operation) for touching a predetermined position on the display surface 141 is performed. When detected, the user interface image UI may be displayed on the display surface. 9A to 9C, all the control units 11 display the same user interface image UI. However, different user interface images may be displayed depending on each operation method. For example, the control unit 11 displays the user interface image UI1 when a front surface operation (second operation) is performed after detecting a back surface operation (first operation). In addition, when the back surface operation (first operation) is detected after the front surface operation (second operation), the user interface image UI2 is displayed. Further, when the back surface operation (first operation) is detected during the period when the front surface operation (second operation) is being performed, the user interface images UI1 and UI2 or user interface images different from these are displayed.

(Modification 4)
In the embodiment, when the control unit 11 displays the user interface image UI on the display surface 141 and then detects that the position on the back touchpad 121 corresponding to the display position of the user interface image UI is operated, The display of the user interface image was stopped. Instead, the control unit 11 displays the user interface image UI in a size smaller than that before the operation, displays the user interface image UI as a translucent image, or displays the user interface image UI near the end of the display surface 141. As described above, the display mode may be changed from that before the operation.

(Modification 5)
In the display device 10 according to the above-described embodiment, the front surface and the back surface face in opposite directions. The positional relationship between the front surface and the back surface of the display device 10 is not limited to this, and may be a direction in which the normal line on the front surface has a component in the normal direction on the back surface. It is included in the meaning that it is provided on the side opposite to the display surface of the surface.
Moreover, although the display apparatus 10 has detected surface operation and back surface operation with contact operation, you may detect operation with respect to the surface or back surface by non-contact proximity operation. Further, the display device 10 may detect the surface operation not by a contact operation but by a physical key pressing operation or the like. In this case, the display device 10 may not include the surface sensor 122.
In the above-described embodiments and modifications, each function realized by the control unit 11 of the display device 10 can be realized by a combination of a plurality of programs, or can be realized by cooperation of a plurality of hardware resources. The display device of the present invention can also be grasped as a program or content display control method executed by the computer or the control unit 11.

DESCRIPTION OF SYMBOLS 10 ... Display apparatus, 11 ... Control part, 111 ... Operation detection part, 112 ... Display control part, 12 ... Operation part, 121 ... Back surface touchpad, 122 ... Front surface sensor, 13 ... Communication part, 14 ... Display part, 141 ... Display surface, 15... Storage unit.

Claims (6)

A display surface for displaying an image;
An operation surface provided on the opposite side of the display surface;
Operation detecting means for detecting an operation performed by a user on the operation surface;
When an operation is detected by the operation detection means, a user interface for the user to instruct a process performed by the user apparatus at a position on the display surface corresponding to the position on the operation surface where the operation is performed Display control means for displaying an image, which is displayed on the first user interface image for processing the content displayed on the display surface or the display surface in response to an operation on the operation surface. Display control means for displaying on the display surface any one of the second user interface images that are not targeted for processing . The display control means causes the display surface to display either the first user interface image or the second interface image according to the position on the operation surface where the operation is performed.
The display device according to claim 1. The display control means includes
Either the first user interface image or the second interface image is displayed on the display surface in accordance with the magnitude of the force when the user performs an operation on the operation surface.
The display device according to claim 1. The display control means includes
Either the first user interface image or the second interface image is displayed on the display surface in accordance with an operation amount when the user performs an operation on the operation surface.
The display device according to claim 1. Detecting an operation performed by a user on an operation surface provided on a side opposite to a display surface for displaying an image;
When an operation on the operation surface is detected, a user interface image for the user to instruct a process performed by the user device at a position on the display surface corresponding to the position on the operation surface where the operation is performed In response to an operation on the operation surface, a first user interface image whose content is displayed on the display surface or a content displayed on the display surface Displaying any one of the second user interface images not to be processed on the display surface;
A display control method comprising: On the computer,
Detecting an operation performed by a user on an operation surface provided on a side opposite to a display surface for displaying an image;
When an operation on the operation surface is detected, a user interface image for the user to instruct a process performed by the user device at a position on the display surface corresponding to the position on the operation surface where the operation is performed In response to an operation on the operation surface, a first user interface image whose content is displayed on the display surface or a content displayed on the display surface Displaying any one of the second user interface images not to be processed on the display surface ;
A program for running

Images