JP5471137B2 - Display device, display method, and program - Google Patents

Description

  The present invention relates to a display device, a display method, and a program. More specifically, the present invention relates to a display device, a display method, and a program for detecting an operation performed by a user on a display screen by a device and controlling display of the display screen according to the detected operation.

  In recent years, mobile devices equipped with a transmissive touch panel are increasing. Many of these devices are equipped with a display as large as possible, and user interface technology is also regarded as important so that an intuitive operation can be performed using a touch panel provided on the display. However, when a button displayed on the display is operated with a user's finger, it is necessary to represent the button to a certain size or more in order to reduce operation mistakes. For this reason, it is difficult to display many GUI (Graphical User Interface) buttons in a mobile device that has a limit in display size.

  On the other hand, by using a device that can detect the contact position and pressure of a finger, even when the user presses one button, a technique for allocating a plurality of different commands according to the pressing force is proposed. (For example, refer to Patent Document 1). With this technology, there is a possibility that the variation of operation will be expanded.

  In addition, a technique has been proposed in which a button that is operated by a user is highlighted when a finger approach is detected (see, for example, Patent Document 2). With this technology, since it is possible to confirm which button the user is trying to press, there is a possibility that erroneous operations are reduced.

JP 2006-251927 A JP 2009-75656 A

  However, in Patent Document 1, when the user presses the button, the pressure must be adjusted by himself, and the operation load is high for the user. In addition, since it is necessary to sense the pressing force, the structure of the device is complicated.

  In Patent Document 2, different display control can be performed by adding different interactions between the proximity state and the contact state. However, the user can clearly distinguish between the proximity state and the contact state and perform a screen operation. Is difficult and may increase operational errors. Further, if this technology is applied to a mobile device, the size of the display on the mobile device is limited as described above, and accordingly the size of the GUI button must be reduced accordingly. In addition to this, since noise is large, it is difficult to use proximity detection and contact detection as triggers for different interactions with the same button.

  In view of the above problems, the present invention provides a display device that changes display of a display screen in accordance with a change in state of an operation performed by the user, in addition to determining whether the user performs an operation performed on the display screen. A display method and program are provided.

  In order to solve the above problems, according to an aspect of the present invention, a display screen, a device that detects an operation performed by a user on the display screen, and the display screen based on a user operation detected by the device are displayed. A state discriminating unit that discriminates one of a contact state, a proximity state, and a remote state, a state change identifying unit that identifies a state change of an operation performed by the user according to the state discriminated by the state discriminating unit, and a user An operation command setting unit configured to set an operation command according to a state change identified by the state change identification unit when setting an operation command for controlling display of the display screen with respect to an operation performed by A display device is provided.

  According to this, when setting the operation command for controlling the display on the display screen for the operation performed by the user, the operation command is set according to the state change identified by the state change identifying unit. Thereby, the variation of screen operation can be increased or operativity can be improved.

  You may further have a screen control part which changes the display of the said display screen according to the operation command set by the said operation command setting part.

  The state change identifying unit identifies, based on the determination result of the state determining unit, whether the operation performed by the user on the display screen has changed from the contact state, the proximity state, or the far state, and The operation command setting unit may set a different operation command according to the state change identified by the state change identification unit.

  A storage unit that stores a previous determination result by the state determination unit; and the state change identification unit is based on the previous determination result stored in the storage unit and the current determination result by the state determination unit. The state change from the contact state to the distant state with respect to the display screen or the state change from the contact state to the close state to the display screen is identified, the operation command setting unit is the state change identified by the state change identification unit Different operation commands may be set according to the operation.

  When the state change identified by the state change identification unit is a state change from a contact state to a far state, the operation command setting unit may prompt the screen control unit to execute the set operation command.

  When the state change identified by the state change identifying unit is a state change from the contact state to the proximity state, the operation command setting unit prompts the screen control unit to postpone execution of the set operation command. Also good.

  In order to solve the above-described problem, according to another aspect of the present invention, a step of causing a device to detect an operation performed by a user on a display screen, and a contact with the display screen based on a user operation detected by the device. Determining a state of a state, a proximity state, and a distant state; identifying a change in state of an operation performed by a user according to the determined state; and And setting an operation command according to the identified state change when setting an operation command for controlling display.

  In order to solve the above-described problem, according to another aspect of the present invention, a process for causing a device to detect an operation performed by a user on a display screen, and a contact with the display screen based on a user operation detected by the device. A process for determining a state, a proximity state, and a distant state, a process for identifying a change in state of an operation performed by the user according to the determined state, and a display screen for the operation performed by the user. When setting an operation command for controlling display, a program for causing a computer to execute a process of setting an operation command according to the identified state change is provided.

  As described above, according to the present invention, the display screen display is changed according to the change in the state of the operation performed by the user, in addition to the determination of the contact and non-contact with the operation performed by the user on the display screen. Can do.

It is the figure which showed the mobile device which concerns on 1st and 2nd embodiment of this invention. It is a figure for demonstrating the detection of the state of the finger | toe by the mobile device which concerns on 1st and 2nd embodiment. It is a functional block diagram of the mobile apparatus which concerns on 1st and 2nd embodiment. It is the flowchart which showed the operation command determination process (main routine) performed with the mobile device which concerns on 1st Embodiment. It is the flowchart which showed the state identification process (subroutine) performed with the mobile device which concerns on 1st Embodiment. FIG. 5 is a diagram for explaining a relationship between a single tap determined by the operation command determination process of FIG. 4 and a finger state change. FIG. 5 is a diagram for describing a relationship between a double tap determined by the operation command determination process of FIG. 4 and a finger state change. It is a figure for demonstrating the other operation command allocated with the mobile device which concerns on 1st Embodiment. It is the flowchart which showed the operation command determination process (main routine) performed with the mobile device which concerns on 2nd Embodiment. It is a figure for demonstrating the relationship between the drag and drop determined by the operation command determination process of FIG. 9, and the state change of a finger | toe. It is the figure which showed an example of the mobile device which concerns on a modification.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

  Note that mobile devices to which the display devices according to the embodiments of the present invention described below are applied are mainly portable terminals such as mobile phones, portable music players, and personal digital assistants (PDAs). It is preferable. However, the display device according to the present invention can also be used in an information processing apparatus such as a notebook PC (Personal Computer) or a stationary PC.

Below, it demonstrates in the following order.
<First Embodiment>
[1-1. Hardware configuration of mobile device]
[1-2. Functional configuration of mobile device]
[1-3. Operation of mobile device: Operation command determination processing]
(Single tap or double tap assignment)
(Assignment of tap or week tap)
Second Embodiment
[2-1. Functional configuration of mobile device]
[2-2. Operation of mobile device: Operation command determination processing]
(Drag and drop execution and execution delay)
<Modification>

<First Embodiment>
[1-1. Hardware configuration of mobile device]
First, the hardware configuration of the mobile device according to the first embodiment will be described with reference to FIG. The mobile device 10 according to the present embodiment is a device having a transmissive touch panel on which a sensor capable of detecting the position of a user's finger is mounted on a display 12. The mobile device 10 can receive various contents via a network (not shown) and display them on the display 12.

  As the display 12, for example, a liquid crystal display (LCD: Liquid Crystal Display), an organic electroluminescence display (organic EL, OELD: Organic Electroluminescence Display), or the like can be used.

  A touch panel 14 is provided on the display 12. The touch panel 14 detects screen operations performed by the user with respect to various GUI buttons 16 displayed on the display 12. The touch panel 14 detects the position of the finger in the x direction and the y direction with respect to the display 12 and detects the state of the finger in the Z direction. As shown in FIG. 2, the touch panel 14 can measure the contact state and proximity state of the finger on the display 12 in a transmissive manner.

  For example, when the distance T from the finger to the surface of the display 12 is larger than a predetermined threshold Th, the touch panel 14 does not detect anything. At this time, it is determined that the finger is far from the display 12.

  In addition, when the distance T from the finger to the surface of the display 12 is smaller than a predetermined threshold Th and the distance T is not 0, the touch panel 14 detects that the finger exists in the proximity region. At this time, it is determined that the finger is in proximity to the display 12.

  Further, when the distance T from the finger to the surface of the display 12 is 0, the touch panel 14 detects that the finger is in contact with the display 12. At this time, it is determined that the finger is in contact with the display 12.

  Note that the touch panel 14 mounted in the present embodiment and the touch panel mounted in the second embodiment and modification described below may be electrostatic or optical as long as they are transmissive. Good. The touch panel 14 corresponds to a device that is provided on the display 12 and detects a screen operation performed by the user on the display 12.

  In the mobile device 10, the microprocessor 30 having the CPU 32, the RAM 34, and the nonvolatile memory 36 shown in FIG. 1 is embedded. Information relating to the screen operation detected by the touch panel 14 is temporarily stored in the various memories. The CPU 32 controls an operation command determination process described later. The following hardware configuration can also be referred to as a hardware configuration of a display device mounted on a mobile device.

[1-2. Functional configuration of mobile device]
The mobile device 10 according to the first embodiment has a functional configuration indicated by the functional blocks shown in FIG. That is, the mobile device 10 according to the present embodiment includes a state determination unit 20, a storage unit 22, a state change identification unit 24, an operation command setting unit 26, and a screen control unit 28.

  Based on the user operation detected by the touch panel 14, the state determination unit 20 determines whether the operation state of the finger is a contact state with the display 12, a proximity state, or a far state.

  The storage unit 22 stores the determination result by the state determination unit 20. Specifically, the storage unit 22 stores whether each determination result is in a contact state, a proximity state, or a far state. In the following description, the previous user operation state is indicated by P (t−1), and the current user operation state is indicated by P (t).

  The state change identifying unit 24 identifies the state change of the operation performed by the user according to the state determined by the state determining unit 20. Specifically, the state change identifying unit 24 determines whether the operation performed by the user on the display 12 has changed from the contact state, the proximity state, or the far state based on the determination result of the state determination unit 20. Identify. Based on the previous determination result stored in the storage unit 22 and the current determination result by the state determination unit 20, the state change identification unit 24 changes the state of the display 12 from the contact state to the far state or the contact state of the display 12. Identifies state changes from to proximity.

  The operation command setting unit 26 sets an operation command for controlling the display on the display 12 for an operation performed by the user. At that time, the operation command setting unit 26 sets an operation command according to the state change identified by the state change identifying unit 24.

  The screen control unit 28 changes the display on the display 12 according to the state change identified by the state change identifying unit 24 in accordance with the operation command set by the operation command setting unit 26.

  The command to each functional component of the mobile device 10 described above is executed by the CPU 32 that executes a dedicated control device or program. A program for executing the operation command determination process described below is stored in advance in the RAM 34 and the nonvolatile memory 36. The CPU 32 executes each function of the state determination unit 20, the storage unit 22, the state change identification unit 24, the operation command setting unit 26, and the screen control unit 28 by reading and executing each program from these memories.

[1-3. Operation of mobile device]
Next, operation command determination processing of the mobile device 10 according to the present embodiment will be described with reference to FIG. FIG. 4 is a flowchart showing an operation command determination process executed by the mobile device 10 according to the present embodiment. The following operations can also be referred to as operations of the display device mounted on the mobile device 10 according to the present embodiment.

  When the operation command determination process is started from step S400, the state determination unit 20 executes a state determination process routine (FIG. 5) in step S405. The state determination process starts from step S500 in FIG. The state determination unit 20 determines whether the user's finger is in a non-contact state and in a proximity state with respect to the display 12 based on the user operation detected by the touch panel 14 in step S505. As described above, when the distance P from the display 12 to the finger is larger than the given threshold Th, it is determined that the finger is in the remote state of FIG. In this case, the process proceeds to step S510, where the state determination unit 20 determines that the current operation state P (t) is the far state, and returns from step S595 to step S410 in FIG.

  On the other hand, if “YES” is determined in the step S505, the process proceeds to a step S515, and the state determining unit 20 determines whether the finger is in contact with the display 12. As described above, if the distance P from the display 12 to the finger is equal to 0, the position of the finger is determined as the contact state in FIG. In this case, the process proceeds to step S520, where the state determination unit 20 determines that the current operation state P (t) is a contact state, and returns from step S595 to step S410 in FIG.

  On the other hand, if NO is determined in step S515, the process proceeds to step S525, in which the state determination unit 20 determines that the current operation state P (t) is the proximity state, and returns from step S595 to step S410 in FIG.

  In step S410, the storage unit 22 stores the operation state P (t) determined this time. Next, proceeding to step S415, the state change identifying unit 24 identifies whether the state of the finger has changed from the contact state to the proximity state. Specifically, the state change identification unit 24 is in the state where the previously determined operation state P (t−1) stored in the storage unit 22 is in the contact state and the operation state P (t) determined this time is close. Determine if it is in a state.

  If the state change is from the contact state to the proximity state, the process proceeds to step S420, the state determination process of FIG. 5 is executed again, and the process proceeds to step S425. In step S425, the state change identifying unit 24 identifies whether the state of the finger has changed from the close state to the far state. Specifically, the state change identifying unit 24 indicates that the previously determined operation state P (t−1) stored in the storage unit 22 is in the proximity state, and the operation state P (t) determined this time is far. Determine if it is in a state.

  As a result of the identification, if the state changes from the contact state to the proximity state (S415) and then changes from the proximity state to the far state (S425), the process proceeds to step S430. The operation command setting unit 26 determines that it is a single tap in step S430, and sets a screen operation operation command that is assigned in advance to the single tap.

  As a result of the identification, if the state does not change from the proximity state to the distant state (S425) after the state change from the contact state to the proximity state (S415), the process proceeds to step S435. The operation command setting unit 26 determines that the operation is a double tap in step S435, and sets an operation command for a screen operation that is assigned in advance to the double tap.

  In step S415, when the state change identifying unit 24 identifies that the state has not changed from the contact state to the proximity state, the process proceeds to step S440, where the operation command setting unit 26 operates the operation command for changing the display on the display 12. The current state is maintained without setting.

(Single tap or double tap assignment)
As a specific example of the operation command determination process described above, an example in which a single tap or a double tap is assigned as an operation command according to a difference in state change will be described with reference to FIGS. 6 and 7.

(Single tap)
As shown in FIG. 6, when the finger operation on the display 12 repeats a distant state → proximity state → contact state → proximity state → distant state... Multiple times, the first state change (distant state → proximity state) The state change identifying unit 24 determines NO in step S415, and proceeds to step S440. The operation command setting unit 26 maintains the current state without setting an operation command for changing the display on the display 12 in step S440.

  Even in the next state change (proximity state → contact state) shown in FIG. 6, the state change identifying unit 24 determines NO in step S415 and proceeds to step S440. The operation command setting unit 26 maintains the current state without setting an operation command for changing the display on the display 12 in step S440.

  In the next state change (contact state → proximity state) shown in FIG. 6, the state change identifying unit 24 determines YES in step S415, and executes the state determination process of FIG. 5 again in step S420. The next state shown in FIG. 6 is a distant state. Therefore, in step S425, the state change identifying unit 24 identifies that the state change is in the proximity state → the far state, and proceeds to step S430. The operation command setting unit 26 determines that the operation is a single tap, and sets an operation command for a screen operation assigned in advance to the single tap. For multiple single taps, the above operation is repeated to repeatedly set operation commands for screen operations previously assigned to the single tap.

  For example, when the screen operation previously assigned to the single tap on the GUI button 16 in FIG. 6 is the next music selection, the operation command setting unit 26 sets an operation command for selecting the next music. The screen control unit 28 displays the next song name selected on the display 12 in accordance with the operation command set by the operation command setting unit 26. By repeating the above operation for a plurality of single taps, music names are displayed on the display 12 one after another in the order of music.

(Double tap)
As shown in FIG. 7, when the operation of the finger on the display 12 is a distant state → proximity state → contact state → proximity state → contact state → proximity state → distant state, the first state change (far state → proximity state) In the state), the state change identifying unit 24 determines NO in step S415 and proceeds to step S440. The operation command setting unit 26 maintains the current state without setting an operation command for changing the display on the display 12 in step S440.

  Even in the next state change (proximity state → contact state) shown in FIG. 7, the state change identification unit 24 determines NO in Step S415 and proceeds to Step S440. The operation command setting unit 26 maintains the current state without setting an operation command for changing the display on the display 12 in step S440.

  In the next state change (contact state → proximity state) shown in FIG. 7, the state change identifying unit 24 determines YES in step S415, and executes the state determination process of FIG. 5 again. The next state shown in FIG. 7 is a contact state. Therefore, in step S425, the state change identifying unit 24 identifies that the state change is the proximity state → the contact state, and proceeds to step S435. The operation command setting unit 26 determines that the operation is a double tap, and sets an operation command for a screen operation assigned in advance to the double tap.

  For example, when the screen operation previously assigned to the double tap for the GUI button 16 in FIG. 7 is the next album selection, the operation command setting unit 26 sets an operation command for selecting the next album. The screen control unit 28 displays the next album name selected on the display 12 in accordance with the operation command set by the operation command setting unit 26.

  Note that the screen control unit 28 may control the screen so that when the GUI button 16 is pressed once, the facial expression of the GUI button 16 changes until the finger leaves the proximity area, as shown in FIG. For example, the screen control unit 28 may change the color of the GUI button 16 to a conspicuous color such as red while the finger is performing a double tap operation.

  As described above, according to the mobile device 10 according to the present embodiment, when the display screen display is controlled according to the contact state detected by the touch panel 14, the setting is made according to the state change next to the proximity state. You can change the operation command. As a result, it is possible to change the display on the display 12 in accordance with the state change in accordance with the set operation command. This can increase variations in screen operations, such as giving a plurality of operation commands to one GUI button, for example, while reducing operability and increasing operability for the user.

  Further, according to the mobile device 10 according to the present embodiment, a user interface capable of high-speed operation can be provided. For example, a conventional double tap has been identified by observing a change in contact state in the time direction. Therefore, it takes a certain time for identification, and a single tap cannot be input a plurality of times in a short time. However, according to the mobile device 10 according to the present embodiment, since the double tap is identified only by the state change, a time factor is not required for identifying the double tap. Thereby, in this embodiment, a fixed time is not necessarily required for identification as in the prior art, and a double tap and a single tap can be input at high speed in a short time.

(Assignment of tap or week tap)
As another example of increasing variations in screen operation, an example in which a tap or a weak tap is assigned as an operation command according to a difference in state change will be described with reference to FIG.

(Tap)
In this example, when step S405 to step S425 of the operation command determination process of FIG. 4 are executed and the process proceeds to step S430, the operation command setting unit 26 determines that the user's screen operation is a normal tap. The operation command setting unit 26 sets scrolling in units of pages as an operation command corresponding to the tap. The screen control unit 28 executes scrolling in units of pages of the screen corresponding to the finger tap operation, as shown in the lower right of FIG. 8 according to the set operation command.

(Week tap)
On the other hand, when step S405 to step S425 of the operation command determination process of FIG. 4 are executed and the process proceeds to step S435, the operation command setting unit 26 determines that the user's screen operation is a weak tap. The operation command setting unit 26 sets scrolling in units of lines as an operation command corresponding to the weak tap. As shown in the lower left of FIG. 8, the screen control unit 28 performs screen-by-line scrolling corresponding to the finger tap operation of the finger in accordance with the set operation command. Thereby, the user can perform rough alignment with a tap, and can perform fine adjustment with a weak tap.

  In the present embodiment, since the tapped coordinates are defined only in the contact state, the position error in the proximity state is not a problem here. In addition, in a device that can measure the proximity state in multiple stages, it is possible to define more than two types of taps.

  As described above, according to the mobile device 10 according to the present embodiment, the state change identification unit 24 changes the state of the display 12 from the contact state to the distant state (transition from the contact state to the distant state through the proximity state) or A state change from the contact state to the proximity state with respect to the display 12 (transition from the contact state to the proximity state and not from the proximity state to the far state) is identified. Then, the operation command setting unit 26 sets a different operation command according to the state change identified by the state change identification unit 24. As described above, in this embodiment, the operation commands are not separately assigned to the contact state and the proximity state, but are separately assigned according to the state change after the contact state. Therefore, it is possible to increase the variation of the screen operation while improving the operability for the user by avoiding the difficulty of the operation when the user performs the screen operation by distinguishing the contact state and the proximity state and the occurrence of an operation error.

Second Embodiment
Next, the mobile device 10 according to the second embodiment will be described. Since the hardware configuration of the mobile device 10 according to the present embodiment is the same as that of the mobile device 10 according to the first embodiment, the description thereof is omitted here.

[2-1. Functional configuration of mobile device]
The mobile device 10 according to the second embodiment has a functional configuration indicated by the functional blocks shown in FIG. That is, the mobile device 10 according to the present embodiment includes a state determination unit 20, a storage unit 22, a state change identification unit 24, an operation command setting unit 26, and a screen control unit 28, as in the first embodiment. Since the functions of the state determination unit 20, the storage unit 22, and the state change identification unit 24 are the same as those in the first embodiment, the description thereof is omitted here. Also in this embodiment, each function of the mobile device 10 is realized by the CPU 32 reading out and executing each program from the RAM 34 and the nonvolatile memory 36.

  The operation command setting unit 26 sets the operation command when the state change identified by the state change identification unit 24 is a state change from the contact state to the far state (transition from the contact state to the far state through the proximity state). Is executed to the screen control unit 28. In response to this, the screen control unit 28 controls display on the display 12 in accordance with the operation command.

  In the operation command setting unit 26, the state change identified by the state change identifying unit 24 is a state change from the contact state to the proximity state (transition from the contact state to the proximity state and not from the proximity state to the far state). In this case, the screen control unit 28 is prompted to postpone execution of the set operation command. In response to this, the screen control unit 28 cancels the execution of the operation command and maintains the current display state of the display 12.

[2-2. Operation of mobile device: Operation command determination processing]
Next, operation command determination processing of the mobile device 10 according to the present embodiment will be described with reference to FIG. FIG. 9 is a flowchart showing an operation command determination process executed by the mobile device 10 according to the present embodiment. In the following, the operation command determination process will be described by taking the execution of drag and drop and the execution as an example. Note that the following operations can also be referred to as operations of the display device mounted on the mobile device 10 according to the present embodiment.

(Drag and drop execution and execution delay)
When the operation command determination process is started from step S900, the state determination unit 20 executes a state determination process routine (FIG. 5) in step S405. Since the state determination process has been described in the first embodiment, it is omitted here. The operation state P (t) determined this time in the state determination process in step S405 is stored in the storage unit 22 in step S410.

  Next, proceeding to step S415, the state change identifying unit 24 identifies whether the state of the finger has changed from the contact state to the proximity state. If the state change is from the contact state to the proximity state, the process proceeds to step S420, the state determination process of FIG. 5 is executed again, and the process proceeds to step S425. In step S425, the state change identifying unit 24 identifies whether the state of the finger has changed from the close state to the far state. As a result of the identification, if the state changes from the contact state to the proximity state (S415) and then further changes from the proximity state to the far state (S425), the process proceeds to step S905, where the operation command setting unit 26 touches the finger. Set the operation command to drag and drop the image.

  On the other hand, as a result of the identification, if the state changes from the contact state to the proximity state (S415) and then does not change from the proximity state to the far state (S425), the process proceeds to step S910, and the operation command setting unit 26 Refrain from performing drag and drop operations on touched images. That is, the image touched by the finger is not dragged and dropped, and the current state is maintained.

  In step S415, when the state change identifying unit 24 identifies that the state has not changed from the contact state to the proximity state, the process proceeds to step S440, where the operation command setting unit 26 operates the operation command for changing the display on the display 12. The current state is maintained without setting.

  According to the operation command determination processing according to the present embodiment, as shown in the lower part of FIG. 9, after the state changes from the contact state to the proximity state, if the state does not change from the proximity state to the far state, the image touched by the finger Do not drag and drop. As described above, in the operation command determination process according to the present embodiment, when the user accidentally lifts his / her finger from the display 12, the drop operation of the dragged data is not executed.

  On the other hand, according to the operation command determination process according to the present embodiment, when the state changes from the contact state to the close state and then changes from the close state to the far state, the image touched by the finger is dragged and dropped. As described above, in the operation command determination process according to the present embodiment, when the user lifts the finger away from the display 12 over the proximity region, it is determined that the finger operation is not a malfunction and the drag-and-drop operation is performed. The

  Conventionally, as shown in the upper part of FIG. 10, even if the user does not intend, if a finger is released from the display 12 even during operation, a drag-and-drop operation is performed, resulting in an operation error. However, in this embodiment, as shown in the lower part of FIG. 10, the state is not dropped when the finger is slightly separated from the display 12 during the operation, and the state change from the close state to the far state is regarded as the drop timing. Thus, it is possible to compensate for an operation error that has occurred in the past and improve the operability of the user.

<Modification>
Finally, a mobile device 50 in FIG. 11 will be briefly described as a modification of the portable device in which the display device of each embodiment described above is mounted. The mobile device 50 has two screens, an upper display 52U and a lower display 52D. Touch panels 58U and 58D are mounted on the upper display 52U and the lower display 52D. The mobile device 50 can be folded by a central hinge mechanism 59 and has a configuration convenient for carrying.

  Even when the display device mounted on the mobile device 50 according to the present modification controls display of the display screen according to the contact state of the finger with respect to the GUI button 16 detected by the device, the display device is controlled according to the identified state change. The display can be changed. Thereby, the variation of screen operation can be increased, improving operability.

  According to each embodiment described above, it is possible to discriminate between a double tap and a plurality of single taps by identifying not only a contact state and a non-contact state but also a change in the state of a finger, or two types of taps. Or assisting the drag-and-drop operation. As a result, it is possible to provide a user interface with high operability by reducing operations with high load on the user.

  In the above embodiment, the operations of the respective units are related to each other, and can be replaced as a series of operations and a series of processes in consideration of the mutual relations. Thereby, the embodiment of the display device can be the embodiment of the display method and the embodiment of the program for causing the computer to realize the functions of the display device.

  As a result, the touch panel 14 detects an operation performed by the user on the display 12, and the contact state, the proximity state, or the far state is determined based on the user operation detected by the touch panel 14. Identifying the state change of the operation performed by the user according to the determined state, and setting the operation command for controlling the display of the display 12 for the operation performed by the user. And a step of setting an operation command in response to a change in state.

  In addition, this allows the touch panel 14 to detect the operation performed by the user on the display 12 and whether the state is a contact state, a proximity state, or a remote state based on the user operation detected by the touch panel 14. When setting a process for determining, a process for identifying a change in state of an operation performed by the user according to the determined state, and an operation command for controlling the display on the display 12 for the operation performed by the user, It is possible to provide a program for causing a computer to execute processing for setting an operation command in accordance with the identified state change.

  In this specification, the steps described in the flowcharts are executed in parallel or individually even if they are not necessarily processed in time series, as well as processes performed in time series in the described order. Including processing to be performed. Even in the steps processed in time series, the order can be appropriately changed depending on circumstances.

  As mentioned above, although preferred embodiment of this invention was described in detail, referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this example. It is obvious that a person having ordinary knowledge in the technical field to which the present invention pertains can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present invention.

  For example, an example in which a plurality of operation commands are given to one GUI button is not limited to the examples described in the above embodiments. As another example in which a plurality of operation commands are given to one GUI button, there is a case of setting content selection and index sending operation commands to one GUI button.

  The state change can also be applied to a character input operation command. For example, when inputting “Thank you”, the user changes the state of the finger from the close state to the far state after inputting “Thank”. A blank input is assigned in advance as an operation command to the state change from the close state to the far state. Thus, a blank between “Thank” and “you” is input. Subsequently, the user inputs “you”. In this way, the finger state change may be applied to character input.

  Examples of the contact sensor of the device for detecting the user's screen operation according to the present invention include a touch sensor, a touch panel, a touch screen, a touch pad, and the like having a matrix switch, a resistive film type switch, a surface acoustic wave type switch, and the like. It is done.

  It is more preferable that the touch panel mounted on the mobile device according to the present invention has a function capable of detecting depression (pressing) on the display 12. Thereby, the touch panel 14 can detect four state changes of a distant state, a proximity state, a contact state, and a pressed state, and it is possible to further increase operation variations by combining these state changes.

DESCRIPTION OF SYMBOLS 10, 50 Mobile device 12 Display 14 Touch panel 16 Button 20 State determination part 22 Storage part 24 State change identification part 26 Operation command setting part 28 Screen control part

Claims (6)

A display screen;
A device for detecting an operation performed by a user on the display screen;
A state discriminating unit for discriminating whether the state is a contact state, a proximity state, or a far state to the display screen based on a user operation detected by the device;
A storage unit for storing a previous determination result by the state determination unit;
Based on the previous determination result stored in the storage unit and the current determination result by the state determination unit, the operation performed by the user on the display screen is changed from the contact state, the proximity state, and the far state to any state. A state change identifying unit for identifying whether the state has changed;
When the state change is a state change from the contact state to the proximity state, the state determination unit further determines the state, and based on the determination result, the state change identification unit determines whether the state change from the proximity state to the far state. A display device comprising: an operation command setting unit that identifies a change or a state change other than a state change from a proximity state to a far state, and sets a different operation command according to the identified state change.   The display device according to claim 1, further comprising a screen control unit that changes display of the display screen in accordance with an operation command set by the operation command setting unit.   The operation command setting unit prompts the screen control unit to execute the set operation command when the state change identified by the state change identification unit is a state change from a contact state to a distant state. The display device described.   When the state change identified by the state change identification unit is a state change from a contact state to a proximity state, the operation command setting unit prompts the screen control unit to cancel execution of the set operation command. Item 3. The display device according to Item 2. Causing the device to detect an operation performed by the user on the display screen;
Determining a state of contact with the display screen based on a user operation detected by the device, a proximity state, and a distant state;
Storing the determined state;
Based on the previous determination result stored in the storing step and the current determination result in the step of determining which state, the operation performed by the user on the display screen is a contact state, a proximity state, and a remote state Identifying from which to which state has changed ,
When the state change is a state change from the contact state to the proximity state, the state is further determined, and based on the determination result, other than the state change from the proximity state to the far state or the state change from the proximity state to the far state Identifying a state change and setting different operation commands according to the identified state change. A process for causing the device to detect an operation performed by the user on the display screen;
A process of determining whether the state is a contact state, a proximity state, or a far state to the display screen based on a user operation detected by the device;
Processing for storing the determined state;
Based on the previous determination result stored in the storing process and the current determination result by the process for determining which state, the operation performed by the user on the display screen is in the contact state, the proximity state, and the remote state A process for identifying from which state the state has changed , and
In the case where the state change is a state change from a contact state to a proximity state, in the process of further determining which state, the state is determined, and based on the determination result, whether the state has changed from any Identifying a state change other than the state change from the proximity state to the far state or the state change from the near state to the far state, and setting different operation commands according to the identified state change, A program that causes a computer to execute.

Images