ES2776103T3 - Procedure and apparatus to display graphical user interface depending on a contact pattern of the user - Google Patents

Abstract

A method of displaying a graphical user interface, GUI, on a touch screen (130) comprising a display unit (132) in an apparatus (100) further comprising a touch sensor unit (110) other than a unit (134) of the touch sensor included in the touch screen (130), the touch sensor unit being located on one side of the apparatus (100), the procedure comprising: detecting (S210) a contact of the unit (110) touch sensor; determining (S220) a contact pattern of the detected contact; displaying (S240) the GUI based on the contact pattern; and characterized by detecting, after displaying (S240) the GUI based on the contact pattern, an increase in contact pressure of the touch sensor unit (110), and changing a display location or size of the GUI elements of the according to the pressure of the detected contact and execute a specific application when the pressure rises beyond a predetermined threshold.

Description

DESCRIPTION

Procedure and apparatus for displaying graphical user interface depending on a user contact pattern

Background of the invention

Field of the invention

Exemplary embodiments of the present invention relate to a graphical user interface (GUI) for electronic devices and, in particular, to a method and apparatus for displaying a GUI according to a user contact pattern.

Background description

A touch screen can be used as a display unit and an input unit. Therefore, electronic devices that have a touch screen may not require additional display and input units. Due to this advantage, a touch screen can be widely used for electronic devices of limited size such as, for example, mobile devices, which can also be referred to as portable devices or handheld devices.

Typically, a user can manipulate a touch screen with one hand or both hands to command a desired function or application execution. When a user uses both hands, one hand can grasp a device, and the other hand can touch a device's touch screen. However, when a user uses only one hand, parts of the touch screen can often be hidden by a finger (for example, the thumb) of a hand that they are grasping.

Figure 10A is an illustrative example showing a user's left thumb selecting one of the menu icons displayed on a touch screen. In this example, if a user touches a specific icon (for example, Music Icon) located in the upper right portion of the touch screen, some of the other icons (for example, Game Icon, Display Icon, and Icon Programmer) displayed on the touch screen may be partially or totally hidden by the thumb. Additionally, these hidden icons can be in contact with the thumb, and thus the functions associated with the hidden icons can be performed in an undesirable manner.

FIG. 10B is another illustrative example showing a user's left thumb touching a scroll bar displayed on the touch screen. If the user touches the scroll bar located on the right side of the touch screen, the displayed content (eg scene) may be hidden by the thumb. Additionally, some of the displayed content can be undesirably touched with the thumb and accessed.

An electronic device that has touch sensors can provide electronic device application control only if a user maintains contact with a specific part of the device without using a touch screen or keyboard. Such electronic devices can provide a display screen with a GUI to guide contact-based entries. If the GUI is displayed in a fixed form regardless of a user contact pattern, some positions in the GUI may fail to note / enter the user contact. This may be due to differences in individual hand sizes, finger sizes, and grip shapes. It is difficult to make a GUI adapted for multiple users. If there is no match between a position in the GUI and a point of contact of the user's contact, confusion can result when a user manipulates applications on the electronic device.

Document US-2006/0111093-A1 discloses an apparatus and method for controlling a portable terminal. The apparatus includes a contact sensing unit that senses an area of the outer surface of the portable terminal contacted by a user when the user holds the portable terminal. A recognition unit recognizes a function mode of the portable terminal based on information about the contacted area perceived by the contact sensing unit. A control unit switches a portable terminal to a function mode recognized by the recognition unit.

JP-2008-027183-A discloses an information processor arranged to display an interface that is suitable for an operator with no specific input required from the operator. The information processor includes a flat case that is dimensioned to be held in one hand and is approximately rectangular in shape. A display device has a display face provided on the main surface of the flat box. Sheet-like pressure sensors are arranged on the first and second side faces of the flat box to detect a distribution of contact areas between the first and second side faces and the human hand. A control part controls the content displayed on the display face by the display device based on the outputs of the pressure sensitive sensors.

US-2006/0197750-A1 discloses a handheld electronic device with multiple touch sensing devices. Touch sensing devices can, for example, be selected from touch panels, touch screens or touch sensitive housings.

Summary of the invention

Exemplary embodiments of the present invention provide a method and apparatus for displaying a graphical user interface (GUI) adapted to a user's manipulative hand.

Additional features of the invention will be described in the description that follows, and in part will be apparent from the description, or may be learned by practice of the invention.

According to a first aspect of the present invention there is provided a method according to claim 1.

According to a second aspect of the present invention there is provided an apparatus according to claim 13.

It is to be understood that both the above general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the claimed invention.

Brief description of the drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the invention, and together with the description serve to explain the principles of the invention .

Figure 1A is a block diagram illustrating an internal configuration of an apparatus in accordance with exemplary embodiments of the present invention.

Figure 1B shows an example of a touch sensor unit located on the lateral sides of the apparatus shown in Figure 1A in accordance with exemplary embodiments of the present invention.

FIG. 2 is a flow chart illustrating a method for displaying a GUI dependent on a manipulator hand in accordance with exemplary embodiments of the present invention.

Fig. 3 is a flow chart illustrating an example of a detailed process of the manipulative hand determination step in the GUI display procedure shown in Fig. 2 according to exemplary embodiments of the present invention.

FIG. 4 is a flow chart illustrating another example of a detailed process of the manipulative hand determination step in the GUI display procedure shown in FIG. 2 according to exemplary embodiments of the present invention.

FIG. 5 is a flow chart illustrating another example of a detailed process of the manipulative hand determination step in the GUI display procedure shown in FIG. 2 according to exemplary embodiments of the present invention.

Figure 6A shows an example of a left hand grip and touch sensor unit located on a lateral side of the apparatus in accordance with exemplary embodiments of the present invention.

Figure 6B shows an example of a right hand grip and touch sensor unit located on a lateral side of the apparatus in accordance with exemplary embodiments of the present invention.

Figure 7A shows another example of a left hand grip and touch sensor unit located on a lateral side of the apparatus in accordance with exemplary embodiments of the present invention.

Figure 7B shows another example of a right hand grip and touch sensor unit located on a lateral side of the apparatus in accordance with exemplary embodiments of the present invention.

Figure 8 illustrates an example of a GUI in accordance with exemplary embodiments of the present invention. Figure 9 illustrates another example of a GUI in accordance with exemplary embodiments of the present invention. Fig. 10A is an illustrative example showing a user's left thumb selecting one of the menu icons displayed on a touch screen in accordance with a conventional GUI.

Fig. 10B is another illustrative example showing a user's left thumb touching a scroll bar displayed on a touch screen in accordance with a conventional GUI.

FIG. 11 is a flow chart illustrating a method of displaying a GUI dependent on a manipulator hand in accordance with exemplary embodiments of the present invention.

FIG. 12A illustrates an example of a screen in which menu icons are displayed in an idle screen application by a user contact in accordance with exemplary embodiments of the present invention.

FIG. 12B illustrates an example of a screen in which displayed menu icons are changed in an idle screen application by a new user contact in accordance with exemplary embodiments of the present invention.

Figure 13A illustrates an example of a screen in which function icons are displayed in a camera application by a user contact in accordance with exemplary embodiments of the present invention.

Figure 13B illustrates an example of a screen in which the displayed function icons are changed in a camera application by a new user contact in accordance with exemplary embodiments of the present invention.

FIG. 14A illustrates an example of a screen in which function icons are displayed in an MP3 application by a user contact in accordance with exemplary embodiments of the present invention.

Figure 14B illustrates an example of a screen in which displayed function icons are changed in an MP3 application by a new user contact in accordance with exemplary embodiments of the present invention.

Detailed description of the illustrated embodiments

The invention is more fully described hereinafter with reference to the accompanying drawings, in which exemplary embodiments of the invention are shown. However, the present invention can be made in many different ways and should not be construed as being limited to the exemplary embodiments described herein. Rather, these exemplary embodiments are provided in such a way that this disclosure will be comprehensive, and will fully convey the scope of the invention to those skilled in the art. In the drawings, the size and relative sizes of layers and regions may be exaggerated for clarity. Like reference numerals in the drawings denote like elements.

It will be understood that, although the terms first, second, third, etc. may be used herein to describe various elements, components, regions, layers and / or sections, these elements, components, regions, layers and / or sections should not be limited by these terms. These terms are only used to distinguish one element, component, region, layer, or section from another region, layer, or section. Thus, a first element, component, region, layer or section discussed below could be referred to as a second element, component, region, layer or section without departing from the teachings of the present invention.

Spatially relative terms, such as 'below', 'below', 'lower', 'above', 'upper', and the like, may be used herein for ease of description to describe the relationship of an element. or feature with other elements or features as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if the device in the As figures are turned over, items described as "below" or "below" other items or features would then be oriented "above" the other items or features. Thus, the exemplary term "below" can encompass both an overhead orientation as well as underneath. The device can be oriented differently (rotated 90 ° or in other orientations) and the descriptors spatially relative u stes herein may be interpreted accordingly.

The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting of the present invention. As used herein, the singular forms "a", "an, an" and "the, the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising", when used in this specification, specify the presence of established features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other characteristics, integers, stages, operations, elements, components, and / or groups thereof.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be construed as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless as expressly defined herein.

Additionally, well-known or widely used techniques, elements, structures, and processes may not be described or illustrated in detail to avoid concealing the essence of the exemplary embodiments.

Before explaining exemplary embodiments of the present invention, relevant terminology will be defined for the description below.

A graphical user interface (GUI) can refer to a graphical display provided on a screen (eg, monitor) of an electronic device. The GUI can include at least one window, at least one icon, at least one scroll bar, and any other graphical items used to enter commands to the device by a user. It should be understood that exemplary embodiments of the present invention may include various types of GUIs in various shapes, designs, and configurations.

A manipulative hand may refer to a hand of the electronic device user occupied with manipulating the touch screen of the electronic device. The manipulative hand may include one or more hands by which a touch action is performed on the touch screen. Additionally, the manipulative hand may include one or more hands in contact with the electronic device having a touch sensor. The manipulator hand can be a left hand, right hand, or both hands of the user.

A touch sensor unit or a touch sensor can refer to at least one sensor sensitive to a user's touch. The touch sensor unit can be different from the touch sensors included in the touch screen and the touch sensor unit can generally be located on at least one lateral side of the device. If a user grasps the device, the touch sensor unit can detect contact between a user's hand and the device, create a contact detection signal, and transmit the contact detection signal to a control unit. The touch sensor unit may be composed of at least one touch sensor that can detect the extent of a contact pressure and the location of contact / pressure. Alternatively, the combination of pressure sensors and touch sensors can be used for the touch sensor unit. The touch sensor unit may include a left sensor portion and a right sensor portion, each of which may be comprised of a plurality of sensor components. The touch sensor unit can be formed on the upper side and / or the lower side, or it can be formed on any and / or all sides of the device.

Sensor components can refer to elements that make up the touch sensor unit. Each sensor component can separately detect a user contact. The number of sensor components included in the touch sensor unit can be determined based on the type or size of sensor components.

A group of components can refer to a set of sensor components arranged in sequence between the contact detection sensor components. The component group can be used to create contact pattern information. The location of the component group and the number of sensor components included in a single component group may vary according to the user's grasp.

Hereinafter, exemplary embodiments of the present invention are described in detail with reference to the accompanying drawings.

Figure 1A is a block diagram illustrating an internal configuration of an apparatus in accordance with exemplary embodiments of the present invention.

Referring to FIG. 1A, apparatus 100 can be a mobile communication terminal, a portable terminal such as a personal digital assistant (PDA), a computer, a TV, or any other electronic device that has a touch screen. Apparatus 100 may include a touch sensor unit 110, a memory unit 120, a touch screen 130, and a control unit 140.

The touch sensor unit 110 can detect the contact of a user's hand when a user grasps the apparatus 100. The touch sensor unit 110 can detect the extent of contact pressure and a location of the contact / pressure. The touch sensor unit 110 can be comprised of touch sensors, touch sensors, and / or a combination of pressure sensors and touch sensors. The touch sensor unit 110 may be located on a lateral side of the apparatus 100, but is not limited thereto. In some exemplary embodiments, the touch sensor unit 110 may be located on each side of the apparatus 100.

Figure 1B shows an example of the touch sensor unit 110 located on the lateral sides of the apparatus 100. After detecting the contact of a user's hand, the touch sensor unit 110 can transmit a contact detection signal to the unit. 140 control. The touch sensor unit 110 may include a left sensor portion and a right sensor portion, each of which may be comprised of a plurality of sensor components.

The memory unit 120 can store a variety of programs necessary to perform functions of the apparatus 100 and data created while performing such functions. The memory unit 120 can store data related to a process of assuming a manipulative hand and the contact pattern information.

The touch screen 130 can display information and can receive user input. The touch screen 130 may include a display unit 132 and a touch sensor unit 134.

The display unit 132 may be formed by a liquid crystal display (LCD) or any other suitable type of display. The display unit 132 can provide a variety of graphical information related to the states and operations of the apparatus 100. The display unit 132 can display a GUI depending on a manipulative hand. In some cases, the display unit 132 may display the GUI according to the position of the user's fingers grasping the apparatus 100.

The touch sensor unit 134 can determine whether the user's hands touch the touch screen or not. The touch sensor unit 134 may be comprised of capacitive touch sensors, pressure sensors, and / or any other suitable touch sensitive sensor. When the touch of the user's hand is detected, the touch sensor unit 134 may transmit a touch signal to the control unit 140. The touch signal may include coordinate data representing the user's touch location. In some cases, the touch sensor unit 134 may not be included in the apparatus 100.

Control unit 140 may monitor states and operations of one or more elements of apparatus 100. For example, control unit 140 may receive a contact detection signal from touch sensor unit 110 and may determine the contact pattern of the user using the contact detection signal. Additionally, the control unit 140 may instruct the display unit 132 to display a GUI depending on the user's contact pattern.

FIG. 2 is a flow chart illustrating a method for displaying a GUI dependent on a manipulator hand in accordance with exemplary embodiments of the present invention.

Referring to FIG. 2, the touch sensor unit 110 can detect a user contact (S210). The user contact can be a result of the user grasping the apparatus 100. When the user contact is detected, the touch sensor unit 110 can transmit the contact detection signal to the control unit 140.

The control unit 140 may receive the contact detection signal from the touch sensor unit 110 (S220). Then, the control unit 140 can determine the contact pattern of the user and thereby can determine a manipulative hand (S230). Step S230 is illustrated in detail in Figure 3, Figure 4, and Figure 5.

After step S230, the control unit 140 may instruct the touch screen 130 to display a GUI based on the manipulating hand (S240). The control unit 140 can then decide whether to receive a further contact detection signal from the touch sensor unit 110 (S250). If the control unit 140 decides to receive additional contact detection signals, the procedure for displaying a GUI may return to step S230 to redetermine the user's contact pattern and further redetermine the manipulator hand. The additional contact detection signal can be provided by the touch sensor unit 110 when the user changes a grip shape.

If the control unit 140 does not receive additional contact detection signal from the touch sensor unit 110, the display unit 132 can maintain a current GUI. From there, the user can manipulate the GUI displayed on the touch screen 130 to enter a command to the apparatus 100.

FIG. 3 is a flowchart illustrating an example of a detailed process of step S230 shown in FIG. 2 according to exemplary embodiments of the present invention.

Referring to FIG. 3, the control unit 140 can generate at least one group of components based on the contact detection signal received from the touch sensor unit 110 (S310). As noted above, the component group can refer to a set of sensor components arranged in one or more sequences among the touch-sensing sensor components.

Exemplary embodiments of the sensor components and the component group are illustrated in Figure 6A, Figure 6B, Figure 7A, and Figure 7B.

Figure 6A shows an example of a user holding the apparatus 100 with a left hand grip. The touch sensor unit 110 can be located on one side of the device 100. The left part of the sensor can be located on the left side of the device 100, and the right part of the sensor can be located on the right side of the device 100. Each sensor part may be composed of a plurality of sensor components. The number of sensor components may vary according to a size of the sensor components. For example, the smaller the sensor components are dimensioned, the more sensor components can be arranged on the lateral sides of the apparatus 100. In Figure 6A, for example, the number of the sensor components belonging to each sensor part it can be twenty-three. Components marked between sensor components on the left side of the sensor may represent components that have detected left hand contact. Among the sensor components on the right side of the sensor, the marked components may represent components that have detected contact with the fingers of the left hand (for example, four fingers except the thumb). Contact sensing components can be grouped by their arrangements into sequence. For example, nine components arranged sequentially on the left side of the sensor can be assigned to a group. And, four pairs of two components on the right side of the sensor can be assigned to four groups.

Returning to Fig. 3, after step S310 of generating the group of components, the control unit 140 can create contact pattern information based on the group of components (S320). Therefore, the contact pattern information may be different based on how the user grasps the apparatus 100. The contact pattern information may include, for example, the number of component groups in each sensor part, the location of component groups, the interval between component groups, the number of sensor components in each component group, and / or pressure sensing data for each sensor component.

Referring to Figure 6A, the contact pattern information on the left side of the sensor may contain the following data: a component group consisting of nine sensor components located, for example, from the twelfth sensor component to the twentieth component of sensor. The contact pattern information on the right side of the sensor can contain the following data: four groups of components, each of which is composed of two sensor components located, for example, in the fourth, fifth, ninth, tenth, fourteenth, fifteenth, nineteenth, and twentieth component positions. Three sensor components can be interposed between two groups of adjacent components.

Returning to FIG. 3, the control unit 140 can retrieve stored contact pattern information from the memory unit 120 (S330). The memory unit 120 can store the contact pattern information, and, in general, can store different contact pattern information corresponding to different types of grasp. Contact pattern information stored in memory unit 120 may include, for example, the number of component groups in each sensor part, the location of component groups, the interval between component groups, the number of components. sensor data for each component group, and / or pressure sensing data for each sensor component.

The control unit 140 can subsequently compare the created contact pattern information with the retrieved contact pattern information (S340). For example, the control unit 140 may perform respective comparisons of the number of component groups in each sensor part, the location of component groups, the interval between the component groups, the number of sensor components in each component group. , and / or pressure sensing data for each sensor component.

The control unit 140 may also determine whether the created contact pattern information is within a range associated with the retrieved contact pattern information (S350). When the created information fully corresponds to the retrieved information, the control unit 140 may determine that the created information is within the range associated with the retrieved information. If an allowable range is previously assigned to the contact pattern information stored in the memory unit 120, the created information may be within the allowable range such that it can be determined that the created information is within the range. The allowable range can be respectively assigned to, for example, the number of component groups in each sensor part, the location of component groups, the interval between the component groups, the number of sensor components in each component group, and / or pressure sensing data for each sensor component.

If the created contact pattern information is within the range of the retrieved contact pattern information, the control unit 140 can determine the manipulator hand that corresponds to the created contact pattern information (S360). The memory unit 120 may already have information stored on different manipulator hands according to the different contact pattern information. If the created contact pattern information belongs to a range of the retrieved contact pattern information, the control unit 140 can determine the manipulative hand that corresponds to the created contact pattern information. The determined manipulative hand can be the left hand, or the right hand.

If the created contact pattern information does not belong to the range of the retrieved contact pattern information, the control unit 140 may determine that the manipulator hand is both hands (S370). After determining the manipulating hand, the control unit 140 may return to the previous step S240 of displaying a GUI according to the manipulating hand.

Figure 4 is a flow chart illustrating another example of a detailed process of step S230 shown in Figure 2 in accordance with exemplary embodiments of the present invention.

Referring to FIG. 4, the control unit 140 may generate at least one group of components which may be a set of sensor components arranged in sequence between contact detection sensor components (S410). The control unit 140 can then calculate a number of sensor components contained in each group of components (S420). For example, as indicated in the example shown in Figure 6A, a single group of components on the left side of the sensor can have nine sensor components, and each of the four groups of components on the right side of the sensor can have two sensor components.

After calculating the number of sensor components in each component group, the control unit 140 can determine which component group and sensor part may have the largest number of contact detection sensor components (S430). For example, the largest group of components may be on the left side of the sensor or the right side of the sensor, and therefore the control unit 140 can determine whether the larger sensor part is the left side of the sensor or the left side of the sensor. right of the sensor in step S430. For example, referring to FIG. 6A, the control unit 140 may determine that the largest group of components has nine sensor components and is located on the left side of the sensor. Referring to another example illustrated in Figure 6B, the largest component group may have nine sensor components and may be located on the right hand side of the sensor. Therefore, the control unit 140 can determine that the largest sensor part is the right part of the sensor. Similarly, the largest sensor part can be the left part of the sensor in Figure 7A and the right part of the sensor in Figure 7B.

If the largest sensor part is the left part of the sensor, the control unit 140 can further determine whether the left part of the sensor has additional component groups (S440). Additional component groups can refer to one or more component groups located in the largest sensor part but that is not the largest component group. In Figure 6A, for example, the left part of the sensor, which is the largest sensor part, may have the largest group of components and no additional group of components. However, in Figure 7A, the left part of the sensor, along with the larger group of components, may have an additional group of components made up of three sensor components from the third sensor component to the fifth sensor component.

If there is no additional component group, as shown in FIG. 6A, the control unit 140 can determine that the manipulator hand is the left hand (S450). The largest group of components can then be considered to be in contact with the palm of the left hand. Also, not having an additional set of components may indicate that the thumb of the left hand may not be in contact with the touch sensor unit 110. In these cases, the control unit 140 may determine that a user can manipulate the touch screen 130 with the thumb of the left hand. That is, the user can use his left hand to grasp the apparatus 100 and touch the touch screen 130. The control unit 140 can determine that the manipulator hand is the left hand.

Similar steps can be performed if the manipulating hand is determined to be the right hand. For example, if the largest sensor part is the right part of the sensor, the control unit 140 can determine if the right part of the sensor has additional component groups (S460). If the right part of the sensor does not have an additional component group, as shown in FIG. 6B, the control unit 140 may determine that the manipulator hand is the right hand (S470).

If the right side of the sensor has additional component groups, the control unit 140 can determine that both hands can be the manipulator hand (S480). The presence of additional component groups may indicate that the thumb of the user's grasping hand is in contact with the touch sensor unit 110. The control unit 140 may determine that a user can manipulate the touch screen 130 with the thumb of a hand that is not grasping. Therefore, the control unit 140 determines that the manipulator hand is both hands.

FIG. 5 is a flow chart illustrating another example of a detailed process from step S230 shown in FIG. 2 according to exemplary embodiments of the present invention.

Referring to FIG. 5, the control unit 140 may generate at least one group of components which may be a set of sensor components arranged in sequence between contact detection sensor components (S510). The control unit 140 can calculate the number of component groups contained in each sensor part (S520). In Fig. 6A, Fig. 6B, Fig. 7A, and Fig. 7B, the sensor parts can be the left part of the sensor and the right part of the sensor. In some cases, the control unit 140 can simultaneously calculate the number of component groups on the left side of the sensor and the right side of the sensor.

For example, in Figure 6A, the number of component groups can be one on the left side of the sensor and four on the right side of the sensor. In Figure 6B, the number of component groups can be four on the left side of the sensor and one on the right side of the sensor. In Figure 7A, the number of component groups can be two on the left side of the sensor and four on the right side of the sensor. In Figure 7B, the number of component groups can be four on the left side of the sensor and two on the right side of the sensor.

The control unit 140 can determine whether the number of component groups is three or more on the left side of the sensor and one or less on the right side of the sensor (S530). If the number of component groups is three or more on the left side of the sensor and one or less on the right side of the sensor, the control unit 140 can determine that the manipulator hand is the user's right hand (S540). Three or more groups of components on the left side of the sensor may indicate that at least three fingers, except the thumb, are in contact with the left side of the sensor. A group of components or less on the right side of the sensor may indicate that the palm of the user's right hand is in contact with the right side of the sensor but the thumb may not be in contact. In such cases, the control unit 140 may determine that the user can manipulate the display screen. tap with the thumb of the user's right hand. That is, a right hand of the user can be used to grasp the apparatus 100 and touch the touch screen 130. Accordingly, the control unit 140 can determine that the manipulator hand is the right hand.

Similarly, by determining that the manipulator hand is the left hand, the control unit 140 can determine whether the number of component groups is three or more on the right side of the sensor and whether it is one or less on the left side of the sensor. sensor (S550).

If the answer is no in both steps 530 and 550, the control unit 140 may determine that the manipulator hand is both hands (S570).

Figure 8 illustrates an example of a GUI in accordance with exemplary embodiments of the present invention.

Figure 8 illustrates an exemplary embodiment of a display screen 810 having menu icons and with the left hand determined as the manipulative hand. The control unit 140 may arrange the menu icons from the upper left corner to the lower right corner of the display 810 to correspond to a left thumb movement path. Accordingly, a user can select icons by touching the display screen 810 with their left thumb to perform desired functions corresponding to the selected icons. Since the menu icons are arranged along a left thumb movement path, no thumb icon can be hidden and an unexpected touch of an icon can be avoided.

FIG. 8 also illustrates an exemplary embodiment of a right hand display 820 determined as the manipulator hand. In such cases, the control unit 140 may arrange the menu icons along a path of movement of the right thumb. In some other cases when it is determined that both hands are the manipulator hand, as shown by the display screen 830 in FIG. 8, the control unit 140 may maintain a changeable normal GUI in accordance with the intention of the user.

Figure 9 illustrates another example of a GUI in accordance with exemplary embodiments of the present invention.

Figure 9 illustrates an exemplary embodiment of a display screen 910 having a scroll bar and with the left hand determined as the manipulative hand. The control unit 140 may arrange the scroll bar along the left side of the display 910 to correspond to a path of movement of the left thumb. Consequently, a user can move the scroll bar up and down by dragging it with the user's left thumb. Since the scroll bar can be arranged along the left side, the displayed content may not be hidden by the thumb and an unexpected touch of the displayed content can be avoided.

Figure 9 also illustrates an exemplary embodiment of a display screen 920 having a scroll bar and with the right hand determined as the manipulator hand. In such cases, the control unit 140 may arrange the scroll bar along the right side of the display 920 to correspond to a path of movement of the right thumb. Therefore, a user can move / drag the scroll bar with the user's right thumb without covering or touching the displayed content. Consequently, an unexpected touch to the displayed content while the user drags the scroll bar can be avoided. In some other cases, as shown by the display screen 930 in FIG. 9, the control unit 140 may maintain a changeable normal GUI when it is determined that both hands are the manipulator hand.

FIG. 11 is a flow chart illustrating a method for displaying a GUI based on a manipulator hand in accordance with exemplary embodiments of the present invention. The procedure described with reference to FIG. 11 may be applicable, for example, when a user uses the touch sensor unit 110 to input a command to the apparatus 100.

Referring to FIG. 11, the touch sensor unit 110 can detect a user contact (S1110). The user's contact may correspond to the user's grasp of the apparatus 100. For example, the user may grasp the apparatus 100 with one hand as shown in FIG. 12A and FIG. 12B or with both hands as shown in FIG. 13A and FIG. 13B. When user contact is detected, the touch sensor unit 110 may transmit the contact detection signal, which includes information about the location and pressure of contacts, to the control unit 140.

The control unit 140 may receive the contact detection signal from the touch sensor unit 110 (S1120). Then, the control unit 140 can determine the user's contact pattern depending on the contact detection signal (S1130). Determination of the user's contact pattern may be based on information about contact location and pressure. In some cases, the memory unit 120 may store a list of the user's grasp forms in relation to the location and pressure of the user's contacts. The control unit 140 may retrieve, from the memory unit 120, a specific grasp shape corresponding to the information received about the location and pressure of contacts. For example, control unit 140 may determine that a user's left hand grabs apparatus 100 if two contact detection signals are received. from the left side of the sensor and four contact detection signals from the right side of the sensor, as shown in Figure 12A.

Then, the control unit 140 may instruct the display unit 132 to display a GUI at a specific location on the display unit 132 depending on the contact location of the user (S1140). Specifically, the control unit 140 may first identify the currently running application (before displaying a GUI), and then it may select GUI items that correspond to the currently running application. For example, when an idle screen application is running, the control unit 140 may select menu icons as GUI items for an idle screen. In other cases, if a camera application is running, the control unit 140 may select an icon to take a photo and a scroll bar to zoom in / out to display. After selecting custom GUI items, the control unit 140 may determine a GUI layout pattern based on the currently running application and the user's grasp. For example, referring to FIG. 12A and FIG. 12B, the control unit 140 may recognize an idle screen application as the currently running application and may also determine that the user's left hand is grasping the apparatus 100. The control unit 140 You can then determine a GUI layout pattern such that the menu icons can be arranged near a contact location of at least one of the four fingers (except the thumb) of the user's left hand.

After determining a GUI layout pattern, the control unit 140 may instruct the display unit 132 to display GUI items based on the GUI layout pattern. That is, the previously selected GUI elements can be displayed on the display unit 132 in accordance with the GUI layout pattern.

FIG. 12A and FIG. 12B illustrate two examples of a screen in which menu icons are displayed in an idle screen application in accordance with exemplary embodiments of the present invention. As shown in Fig. 12a and Fig. 12B, three menu icons can be arranged at positions in one direction across from three finger contact locations. The memory unit 120 can store ratings of how frequently the menus and menu icons are used. Control unit 140 may arrange menu icons in order of frequency of use. For example, when a user grabs the apparatus 100 during the execution of an idle screen application, the control unit 140 may retrieve the frequency-of-use ratings menus from the memory unit 120 and may instruct the display unit 132 to display menu icons according to retrieved ratings. The icon display on the display unit 132 can be changed according to the user's preference.

The control unit 140 can also change the location of the GUI elements in accordance with the user's contact pressure. Referring to FIG. 12A and FIG. 12B, when the user increases the pressure (eg, the pressure increases beyond a predetermined pressure threshold) of contacts while gripping the apparatus 100, the menu icons displayed on the unit 132 of display can be moved to the right side of the screen, namely to the pressed fingers. Additionally, the control unit 140 can execute a specific application when the corresponding icon reaches the right side of the display unit 132. In some cases, the control unit 140 may decide whether to execute applications based on the contact pressure of the user's fingers and / or when the icons reach the right side of the display unit 132.

The control unit 140 can also change the display size of GUI elements in accordance with the pressure of a user contact. For example, when the user increases the contact pressure, the menu icons displayed on the display unit 132 may be enlarged or reduced in size. An increase in contact pressure may highlight the menu icons displayed on the display unit 132.

Figure 13A and Figure 13B illustrate two examples of a screen on which function icons are displayed in a camera application in accordance with exemplary embodiments of the present invention. Referring to Figure 13A and Figure 13B, a user can grasp the apparatus 100 with the thumbs and forefingers of both hands. An icon for taking a photo can be located at a position near the index of the right hand, and a scroll bar for zooming in / out can be located at a position in a direction along from the thumb of the right hand. If the user increases the contact pressure using the right index, an icon for taking a photo moves to the right index. When this icon reaches the upper side of the display unit 132, a function of taking a photo can be executed. In addition, the user can increase the contact pressure by using the right thumb to regulate the zoom in / out.

Figure 14A and Figure 14B illustrate two examples of a screen in which function icons are displayed in an MP3 application. Referring to Fig. 14A and Fig. 14B, a user can grasp the apparatus 100 with the left hand. Function icons may be displayed depending on the contact locations of the left hand fingers other than the thumb, while a volume control bar may be displayed depending on the contact location of the thumb. The function icons displayed can follow a predefined GUI layout pattern. While gripping the apparatus 100, the user can regulate the execution of an MP3 application by increasing a contact pressure or by performing an action such as tapping. For example, the location, size, and / or expression effect of the icons may vary according to the user's contact.

Returning to Fig. 11, after displaying a GUI, the control unit 140 can determine whether the contact location of the user has changed (S1150). Specifically, when a user changes a grip shape while gripping the apparatus 100, the touch sensor unit 110 can detect a change in the user's contact and can generate a new contact detection signal. The control unit 140 can then receive the new contact detection signal from the touch sensor unit 110, can determine the user's contact pattern again, and can modify the display of a GUI depending on the new contact pattern.

Referring to Figure 12A and Figure 12B, the user contact in Figure 12B may be different from the user contact in Figure 12A. For example, the location of the fingers may have moved downward in Figure 12B. Control unit 140 may receive a new contact detection signal from touch sensor unit 110 and may determine a new contact pattern based on new information on contact location and pressure. The control unit 140 may then instruct a GUI display to change in accordance with the new contact pattern.

Comparing Figure 13A and Figure 13B, the user's right index may have moved (eg, to the left) in Figure 13B, compared to Figure 13A. Control unit 140 can receive a new contact detection signal, determine a new contact pattern, and move the photo icon to a current index contact, as shown in FIG. 13B.

Referring to Figure 14A and Figure 14B, the number of contacts as well as the location of the contacts may change. For example, four contacts on the right side in Figure 14A can move downward and can be reduced to three contacts in Figure 14B. Also, a contact on the left side can move down. A volume control bar can also be moved down along the left side. Additionally, a back icon, a play / pause icon, and a stop icon can move down along the right side, while a forward icon that corresponds to a finger (for example, smallest finger) of the left hand can be removed from the display unit 132.

As indicated above, exemplary embodiments of the present invention disclose a method and apparatus for displaying and modifying a GUI depending on the location and pressure of a user's contacts. Accordingly, exemplary embodiments of the present invention can avoid confusion when a user manipulates an apparatus.

It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the scope of the invention defined in the appended claims.

Claims (14)

A method of displaying a graphical user interface, GUI, on a touch screen (130) comprising a display unit (132) in an apparatus (100) further comprising a different touch sensor unit (110) of a touch sensor unit (134) included in the touch screen (130), the touch sensor unit being located on a lateral side of the apparatus (100), comprising the procedure: detecting (S210) a contact of the touch sensor unit (110); determining (S220) a contact pattern of the detected contact; displaying (S240) the GUI based on the contact pattern; and characterized by detecting, after displaying (S240) the GUI based on the contact pattern, an increase in contact pressure of the touch sensor unit (110), and changing a location or display size of the GUI elements according to with the detected contact pressure and run a specific application when the pressure rises beyond a predetermined threshold. The method of claim 1, further comprising determining (S230) a manipulative hand based on the contact pattern. The method of claim 2, wherein determining (S220) a contact pattern comprises: generating (S310) one or more groups of components comprising a set of sensor components that detect contact and are arranged in one or more sequences, the set of sensor components is located in at least one sensor part of the unit ( 110) of touch sensor; creating (S320) contact pattern information based on the one or more component groups; comparing (S340) the created contact pattern information with stored contact pattern information; and determining (S350) the manipulator hand that corresponds to the created contact pattern information if the created contact pattern information is within a range (S350) associated with the stored contact pattern information. The method of claim 3, wherein the contact pattern information comprises a number of the one or more groups of components in each sensor part, a location of the one or more groups of components, an interval between the one or more more component groups, a number of sensor components in each component group, and / or pressure sensing data for each sensor component. The method of claim 2, wherein determining (S220) the contact pattern comprises: generating (S410) one or more groups of components comprising a set of sensor components that detect contact and are arranged in one or more sequences, the set of sensor components is located in at least one sensor part of the unit ( 110) of touch sensor; determining (S430) a sensor part having a larger group of components comprising a larger number of sensor components; determining (S440, S460) if the largest component group has additional component groups; and determine (S450, S470), if there are no additional component groups, that the manipulator hand is a left hand of a user when the largest component group belongs to a left part of the sensor of the touch sensor unit (110), or that the manipulator hand is a right hand of the user when the largest group of components belongs to a right part of the sensor of the touch sensor unit (110). 6. The method of claim 2, wherein determining (S220) the contact pattern comprises: generating (S510) one or more groups of components comprising a set of sensor components that detect contact and are arranged in one or more sequences, the set of sensor components is located in at least one sensor part of the unit ( 110) of touch sensor; calculating (S520) a number of component groups contained in each sensor part; determine (S540) that a right hand of the user is the manipulator hand when the number of component groups is three or more in a left part of the sensor of the touch sensor unit and is one or less in a right part of the sensor of the touch sensor unit. touch sensor unit (S530); and determining (S560) that a user's left hand is the manipulating hand when the number of component groups is three or more on the right side of the sensor and one or less on the left side of the sensor (S550). The method of claim 2, wherein displaying (S240) the GUI comprises: providing menu icons on the display unit (132) that correspond to a movement path of a left thumb of a user when the manipulator hand is a left hand of the user; and arranging the menu icons on the display unit 132 that corresponds to a movement path of a user's right thumb when the manipulator hand is the user's right hand. The method of claim 2, wherein displaying (S240) the GUI comprises: disposing a scroll bar along a left portion of the display unit (132) when the manipulator hand is a left hand of a user; and arranging the scroll bar along a right portion of the display unit (132) when the manipulator hand is a right hand of the user. The method of claim 1, wherein determining (S220) the contact pattern comprises determining a form of grasp of a user based on a location of the detected contact. The method of claim 9, wherein determining (S220) the contact pattern further comprises determining whether a contact pressure is greater than a threshold value. The method of claim 9, further comprising: determine a GUI layout pattern according to a currently running application and the user's grasp form. The method of claim 1, wherein displaying (S240) the GUI comprises displaying the GUI at a position on the display unit (132) based on a location of the detected contact. 13. A widget (100) for displaying a graphical user interface, GUI, the widget (100) comprising: a touch screen (130) comprising a display unit (132) configured to display (S240) the GUI, and a touch sensor unit (134); a touch sensor unit (110), other than touch sensor unit (134), located on a side side of apparatus (100) and configured to generate (S210) a contact detection signal in response to detection of a contact, the touch sensor unit (110) comprising a plurality of sensor parts comprising a left sensor part and a right sensor part, each sensor part having a plurality of sensor components; and characterized in that a control unit (140) is configured to receive the contact detection signal from the touch sensor unit (110), to determine (S220) a contact pattern based on the contact detection signal, to indicate to the display unit 132 showing (240) the GUI based on the contact pattern, to detect, after displaying (S240) the GUI based on the contact pattern, an increase in the contact pressure of the unit (110) touch sensor, and to change a location or display size of the GUI elements in accordance with the detected contact pressure and to execute a specific application when the pressure increases beyond a predetermined threshold. The apparatus of claim 13, wherein the control unit (140) is further configured to generate (S310) one or more groups of components that comprise a set of contact-sensing sensor components that are arranged in one or more sequences, to create (S320) contact pattern information corresponding to the contact pattern based on the one or more component groups, to compare (S340) the created contact pattern information with stored contact pattern information , and to determine (S360) a manipulative hand based on the created contact pattern information whether the created contact pattern information is within a range (S350) associated with the stored contact pattern information.