KR20170043076A - Electronic device and method for processing gesture thereof - Google Patents

Abstract

A gesture processing method of an electronic device is disclosed. The method of the present invention comprises the steps of: detecting a hovering gesture in each of a first area and a second area when the hovering gesture is inputted while a flexible display of the electronic device is folded and divided into the first area and the second area based on a folding axis; and determining a hovering point corresponding to the hovering gesture based on a detection position of each of the first area and the second area. Accordingly, the hovering gesture can be more accurately detected than before.

Description

[0001] DESCRIPTION [0002] ELECTRONIC DEVICE AND METHOD FOR PROCESSING THE SAME [0002]

Embodiments of the present disclosure are directed to electronic devices and methods of processing gestures thereof, and more particularly to electronic devices for processing hovering gestures in flexible displays and methods for processing gestures thereof.

With the development of display technology, flexible display and transparent display are being developed. A flexible display means a flexible display device.

In flexible displays, flexible substrates can be folded and unfolded by replacing glass substrates that wrap the liquid crystal with plastic films in conventional LCDs and organic light emitting diodes (OLEDs). Flexible displays have the advantage that they can be bent or bent and can be manufactured in various shapes.

For example, the field of IT products such as cellular phones and ultra-small PCs that can be folded or rolled up, and e-books that can replace magazines, textbooks, books, and comics can be used in flexible displays. have. In addition, since flexible displays use flexible plastic substrates, they can be extended to clothing fashion and medical diagnostics.

On the other hand, with the commercialization of flexible displays, development of various interfaces in a foldable or rolled-up electronic device is being studied using the bent or folded nature of the flexible display.

Embodiments of the present disclosure are directed to providing a method of processing a hovering gesture on an electronic device having a flexible display and on a flexible display, particularly on a folded area where the flexible display is folded.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed.

According to an aspect of the present disclosure, a gesture processing method of an electronic device includes: when a hovering gesture is input when the electronic device is folded so that the flexible display of the electronic device is divided into a first area and a second area with respect to a folding axis , Sensing the hovering gesture in the first region and the second region, respectively, and determining a hovering point corresponding to the hovering gesture based on the sensed position of each of the first region and the second region do.

The hovering gesture may be performed on a folded area comprising the folding axis of the flexible display.

The gesture processing method may further include sensing a folding state of the electronic device, and the folding region may be determined according to a sensed folding state of the electronic device.

The folding state of the electronic device can be an inbending state in which the electronic device is bent inward, or an outbending state in which the electronic device is bent outward.

In addition, the size of the folding area may increase as the folding angle of the electronic device increases.

The sensing position in the first region may be a position of an electrode having a peak point in the sensed value measured for the hovering gesture among the electrodes in the first region.

Alternatively, the gesture processing method of the electronic device may include the steps of detecting at least one hovering position in the folding area when the hovering gesture is input on the folding area including the folding axis with the electronic device folded, And determining a hovering point corresponding to the hovering gesture in the folded area by correcting the position.

Alternatively, the gesture processing method of the electronic device may further comprise the steps of: sensing at least one hovering position in the folding area when the hovering gesture is input on the folding area including the folding axis with the electronic device folded; Determining a hovering point corresponding to the hovering gesture by correcting the sensed hovering position, and displaying the UI at the determined hovering point.

Alternatively, a method of processing a gesture of an electronic device includes the steps of sensing a plurality of hovering positions on a folding area including a folding axis in a state where the electronic device is folded, determining a plurality of hovering points by correcting the plurality of hovering positions And determining a direction of movement of the hovering gesture on the folded area based on the determined plurality of hovering points.

Alternatively, the gesture processing method of the electronic device may include the steps of sensing a hovering gesture using an input tool on a folding area including a folding axis in a state where the electronic device is folded, and determining, based on the movement characteristics of the inputting tool, And determining a moving direction.

Meanwhile, an electronic device according to an embodiment of the present invention includes: A flexible display that can be divided into a first area and a second area based on a folding axis and a hinging gesture that is input when the flexible display is folded, And a processor for determining a hovering point corresponding to the hovering gesture based on the second position.

Here, the hovering gesture may be performed on a folded area including the folding axis of the flexible display.

In addition, the electronic device may further include a state sensing unit for sensing a folding state of the electronic device, and the processor may determine the folding region according to the sensed folding state.

The folding state of the electronic device can be an inbending state in which the electronic device is bent inward, or an outbending state in which the electronic device is bent outward.

Also, the size of the folding area may increase together with the folding angle of the electronic device.

In addition, the first position may be a position of an electrode having a peak point in a sensed value measured according to the hovering gesture among the electrodes in the first region.

Alternatively, the electronic device may be configured to correct at least one hovering position sensed in the folding area, when the hovering gesture is input on the folding area including the flexible display and the folding axis, the flexible display being bent with respect to the folding axis, And a processor for determining a hovering point corresponding to the gesture.

Alternatively, the electronic device may include a flexible display that is flexed relative to the folding axis, and responsive to the hovering gesture determined by correcting at least one hovering position sensed in the folding area, when a hovering gesture is input on the folding area comprising the folding axis And controlling the flexible display so that the UI is displayed at a hovering point at which the UI is displayed.

Alternatively, the electronic device may determine a plurality of hovering points by correcting a plurality of hovering positions sensed in the folding area when a movement of the hovering gesture is made on a flexible display that is bent with respect to the folding axis and on the folding area including the folding axis And determining a movement term of the hovering gesture based on the determined plurality of hovering points.

Alternatively, the electronic device may be configured to move the hovering gesture based on the movement characteristics of the input tool when a movement of the hovering gesture using the input tool is made on a flexible display that is flexed relative to the folding axis and a folding area comprising the folding axis, And < / RTI >

According to embodiments of the present disclosure, an accurate hovering point corresponding to the hovering gesture on the flexible display can be determined.

In particular, depending on the hovering gesture on the folding area where the flexible display is bent, the exact hovering point corresponding to the hovering gesture in the folding area can be determined.

This allows the electronic device to accurately reflect the intent of the user performing the hovering gesture. That is, in response to an accurate hovering point corresponding to the hovering gesture, the user's satisfaction with the electronic device can be increased by performing functions such as selection, execution, or movement of objects on the flexible display.

1 is a diagram for explaining an electronic device according to an embodiment of the present disclosure.
2 is a side view of an electronic device for explaining the degree of detection according to a hovering gesture, in accordance with an embodiment of the present disclosure;
Figure 3 is a diagram showing the degree of detection according to the hovering gesture on the X and Y axes, in accordance with one embodiment of the present disclosure;
4 is a block diagram of an electronic device, in accordance with one embodiment of the present disclosure;
5 is a flow diagram illustrating a method of processing gestures of an electronic device, according to one embodiment of the present disclosure;
6 is a diagram illustrating a method for determining a hovering point in an electronic device in a folded state, in accordance with one embodiment of the present disclosure;
Figure 7 is a diagram illustrating a method by which an electronic device determines a hovering point, in accordance with one embodiment of the present disclosure;
8 is a diagram illustrating a method by which an electronic device determines a hovering point, in accordance with another embodiment of the present disclosure;
Figures 9A and 9B are diagrams illustrating a method by which an electronic device determines a hovering point, in accordance with another embodiment of the present disclosure.
10 is a diagram illustrating a method by which an electronic device determines a hovering point, in accordance with another embodiment of the present disclosure;
11 is a diagram illustrating a method by which an electronic device determines a hovering point, in accordance with another embodiment of the present disclosure;
12 is a diagram illustrating a method by which an electronic device determines a hovering point according to a folding angle, according to another embodiment of the present disclosure;
13 is a side view of an electronic device for explaining an outfolding state, in accordance with one embodiment of the present disclosure;
14 is a flow chart illustrating a method of processing a gesture of an electronic device according to another embodiment of the present disclosure;
Figure 15 is a diagram illustrating a method for determining a hovering point in an electronic device in an outfolded state, in accordance with one embodiment of the present disclosure;
16 is a block diagram of an electronic device, in accordance with another embodiment of the present disclosure;
17 and 18 are flowcharts illustrating a method of processing a gesture of an electronic device according to another embodiment of the present disclosure
19 and 20 are views showing screens in which an electronic device displays a UI in a folded area, according to an embodiment of the present disclosure.
Figure 21 is a diagram illustrating a screen displaying a drawing or handwriting on a folded area of an electronic device, in accordance with one embodiment of the present disclosure;
Figures 22 and 23 are views showing screens for selecting items displayed in the folded area of the electronic device according to an embodiment of the present disclosure.
Figures 24 and 25 are diagrams illustrating a method by which an electronic device recognizes movement of a hovering gesture, in accordance with one embodiment of the present disclosure.
Figures 26 and 27 are diagrams illustrating screens in which an electronic device displays a UI in accordance with movement of a hovering gesture, in accordance with one embodiment of the present disclosure.
Figures 28 to 30 are flow charts illustrating gesture processing methods of an electronic device, according to one embodiment of the present disclosure.

Hereinafter, various embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood, however, that this is not intended to limit the techniques described in this disclosure to particular embodiments but includes various modifications, equivalents, and / or alternatives of the embodiments of the disclosure . In connection with the description of the drawings, like reference numerals may be used for similar components.

In this disclosure, the expressions "having," "having," "including," or "including" may be used to denote the presence of a feature (eg, a numerical value, a function, an operation, Quot ;, and does not exclude the presence of additional features.

In this disclosure, expressions such as "A or B," " at least one of A and / or B, "or" one or more of A and / or B ", may include all possible combinations of the listed items . For example, "A or B," "at least one of A and B," or "at least one of A or B" includes (1) at least one A, (2) Or (3) at least one A and at least one B all together.

As used in this disclosure, the expressions "first," "second," "first," or "second," etc. may modify various components, But is used to distinguish it from other components and does not limit the components.

(Or functionally or communicatively) coupled with / to "another component (eg, a second component), or a component (eg, a second component) Quot; connected to ", it is to be understood that any such element may be directly connected to the other element or may be connected through another element (e.g., a third element). On the other hand, when it is mentioned that a component (e.g., a first component) is "directly connected" or "directly connected" to another component (e.g., a second component) It can be understood that there is no other component (e.g., a third component) between other components.

The phrase " configured to ", as used in this disclosure, means that, depending on the context, for example, "having the capacity to, To be designed to, "" adapted to, "" made to, "or" capable of ". The term " configured to (or set up) "may not necessarily mean" specifically designed to "in hardware. Instead, in some situations, the expression "configured to" may mean that the device can "do " with other devices or components. For example, a processor configured (or configured) to perform the phrases "A, B, and C" may be implemented by executing one or more software programs stored in a memory device or a dedicated processor (e.g., an embedded processor) , And a generic-purpose processor (e.g., a CPU or an application processor) capable of performing the corresponding operations.

The user input in the present disclosure may include, but is not limited to, at least one of a bending input, a voice input, a button input, and a multimodal input. The user input may also include a touch gesture input, which is a contact touch, and a hovering gesture input, which is a contactless touch. The user can select, execute or move objects on the flexible display in a hovering gesture manner. Here, the hovering gesture can be performed using a variety of input tools (e.g., user's fingers, stylus pen, digitizer pen).

An application in this disclosure may be a set of computer programs designed to perform a particular service.

The terms used in this disclosure are used only to describe certain embodiments and may not be intended to limit the scope of other embodiments. The singular expressions may include plural expressions unless the context clearly dictates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by one of ordinary skill in the art. The general predefined terms used in this disclosure may be interpreted as having the same or similar meaning as the contextual meanings of the related art and, unless explicitly defined in the present disclosure, include ideally or in an excessively formal sense . In some cases, the terms defined in this disclosure can not be construed to exclude embodiments of the present disclosure.

Various embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

1 is a diagram for explaining an electronic device 100 according to an embodiment of the present disclosure.

The electronic device 100 may employ a flexible display 130 that can be bent or folded as shown in FIG. For example, the electronic device 100 may include a foldable display that can be folded or unfolded at a particular angle or curvature, a bendable display that can be bent or spread with a particular curvature, It is possible to employ various kinds of flexible displays 130 in which the shape of the display can be deformed by an external force, such as a cylindrical rollable display. The flexible display 130 may be divided into a first area 11 and a second area 12 with respect to a folding line 20. [ Once the folding is done, the folding area 13 comprising the folding axis 20 is determined on the flexible display 130. The folding region 13 may include a portion of each of the first region 11 and the second region 12. The folding area 13 means a peripheral area of the folding axis.

The folding area 13 can be determined in different sizes and shapes depending on the folding state of the electronic device 100, at the periphery of the folding axis. For example, the size of the folded area 13 can be determined according to the folding angle at which the electronic device 100 is bent in or out. In this case, as the folding angle [theta] 1 at which the electronic device 100 is bent inward is increased, the size of the folding area 13 may gradually increase. Alternatively, as the folding angle at which the electronic device 100 is bent outwardly increases, the size of the folding area 13 may gradually increase.

The folding axis 20 may refer to a line where the flexible display 130 is bent or folded. The folding axis 20 may be a line corresponding to a boundary between the first and second regions. That is, when the flexible display 130 is bent, at least a part of the flexible display 130 is deformed. A line connecting the point where the degree of deformation is greatest can be regarded as a folding axis.

 The folding shaft 20 may be, for example, an axis that is bent or folded by the hinge means provided in the electronic device 100. [ For example, the folding axis 20 may be the middle line of the flexible display 130 when the electronic device 100 is folded symmetrically.

Alternatively, the electronic device 100 may not be provided with the hinge means. In this case, the folding axis 20 can be any line that the flexible display 130 bends or folds at various angles and directions (e.g., vertical, horizontal, diagonal, etc.) by external force by the user.

Throughout this specification, "folded state" may refer to a state in which the electronic device 100 is bent in or out of the folding axis 20. The "folding state" may include both "in folding state" and "outfolding state ". 1 (a), a state in which the folding angle [theta] 1 of the electronic device 100 is 180 [deg.] Or substantially 180 [deg.] Is a state in which the electronic device 100 is unfolded Quot; unfolded state ".

In Fig. 1, the electronic device 100 can be converted from an unfolded state in Fig. 1 (a) to an infolding state in Fig. 1 (b), for example.

For the purposes of this specification, "in folding state" may refer to a state in which the electronic device 100 is bent inward relative to the folding axis 20. At this time, the folding angle [theta] 1 of the electronic device 100 may be a value between 0 [deg.] And 180 [deg.]. In this case, the state where the folding angle [theta] 1 is 0 [deg.] Is a state in which the electronic device 100 is completely folded inward with respect to the folding axis 20 so that the mask faces of the first region 11 and the second region 12 face each other Quot; state " or " substantially parallel " state. In this case, the electronic device 100 may be referred to as a "fully folded state"

In Fig. 1, the electronic device 100 can be converted from, for example, the unfolded state in Fig. 1 (a) to the outfolding state in Fig. 1 (c).

Throughout this specification, an "outfold state" may refer to a state in which the electronic device 100 is bent outward relative to the folding axis 20. At this time, the folding angle [theta] 1 of the electronic device 100 may be a value between 180 [deg.] And 360 [deg.]. In this case, the state where the folding angle [theta] 1 is 360 [deg.] Is a state in which the electronic device 100 is completely folded out on the basis of the folding axis so that the back surface of the first area 11 and the back surface of the second area 12 are parallel It can mean almost parallel. In this case, the electronic device 100 may be referred to as "completely outfolded ".

1 (d), the electronic device 100 may have a plurality of folding axes 20, 21. In addition, In this case, the electronic device 100 may have both an in-fold state and an out-fold state. The flexible display 130 may include a first area 11, a second area 12, and a third area 15 based on a plurality of folding axes 20 and 21. In addition, the flexible display 130 may include a plurality of folding areas 13,14 each including a plurality of folding axes 20,21.

1 (a) to 1 (d), a hinge unit is provided in a bezel area surrounding the flexible display 130. FIG. In this case, the number and position of the folding axes may be fixed depending on the number and position of the hinge means. Further, the folding area is determined as a flat area around the folding axis, and the size thereof may vary depending on the degree of folding.

Alternatively, the bezel itself of the electronic device 100 may be formed of a flexible material, or may be implemented as an electronic device having a bezel omitted. Fig. 1 (e) shows such an embodiment. As shown in FIG. 1 (e), when the electronic device 100 is not provided with the hinge means, the flexible display 130 can be bent or folded at various angles and directions by the external force of the user. In this case, the flexible display 130 may have a plurality of folding axes 20, 21, 22 based on a line that is bent or folded by the external force of the user. The flexible display 130 includes a first region 11, a second region 12, a third region 15, and a fourth region 16 on the basis of a plurality of folding axes 20, . In addition, the flexible display 130 may include a plurality of folding areas 13,14, 17 each including a plurality of folding axes 20,21, 22.

In the electronic device having the structure as shown in FIG. 1 (e), the folding axes can be formed at various positions in various numbers depending on the intention of the user. Also, the folding area may be used to mean all of the area where the deformation occurs in the periphery of the folding axis.

Hereinafter, the embodiments of the present disclosure will be described mainly with reference to the case where the electronic device 100 has one folding axis 20, but the embodiments are not limited to those shown in Fig. 1 (d) e), the electronic device 100 can be applied to both cases having a plurality of folding axes.

2 is a side view of an electronic device 100 for explaining the degree of detection according to a hovering gesture in a folded state, in accordance with an embodiment of the present disclosure.

In Fig. 2, the electronic device 100 may be folded inward and into a folded state. In the folded state, the electronic device 100 may sense the hovering gesture of the user using the input tool 30 on the folded area 13 of the flexible display 130. In this case, even if the user takes a hovering gesture at the desired hovering point 40, the detection value of the peak point at that hovering point 40 (e.g., the capacitance value or the magnetic field value) may not be measured. That is, if a hovering gesture is made in the folded state, hovering gestures may be sensed in various areas of the flexible display 130, as shown in FIG. The point where the hovering gesture is sensed in the first region 11 is referred to as a first position 41 and the point where the hovering gesture is sensed in the second region 12 is referred to as a second position 42 ). For example, the detection value of the peak point 51 according to the hovering gesture at the first position 41 of the first region 11 is measured and the peak value 51 at the second position 42 of the second region 12 is measured, The sensing value of the sensor 52 may be measured.

In other words, as the distance from the input tool 30 to the first position 41 and the second position 42 is closer to the distance from the input tool 30 to the hovering point 40, A hovering gesture can be sensed at another first position 41 and second position 42. In this case, a hovering gesture may be sensed in both the first 41 and second 42 positions, or a hovering gesture may be sensed as it crosses the first 41 and second 42 positions .

In this case, the electronic device 100 may determine a hovering point corresponding to the hovering gesture on the flexible display 130 based on the first location 41 and the second location 42. Specific methods for determining the hovering point will be described in detail later with reference to FIG. 3 to FIG.

Figure 3 is a diagram showing the degree of detection according to the hovering gesture in the folded state on the X and Y axes, according to one embodiment of the present disclosure;

In Figure 3, the flexible display 130 may be folded at 90 degrees to have a rounding (curvature) of, for example, 5R. In this case, the graphs 310 and 320 of FIG. 3 represent the sensed values (e.g., capacitance values) measured on the touch panel, according to the hovering gesture of the user.

The touch panel of the flexible display 130 for sensing the hovering gesture may be composed of a plurality of horizontal electrodes and a plurality of vertical electrodes. At this time, a point where the plurality of transverse electrodes and the plurality of longitudinal electrodes intersect is referred to as an electrode junction.

3 (a) shows a transverse electrode of the flexible display 130, and Fig. 3 (b) shows a longitudinal electrode of the flexible display 130. Fig.

In FIG. 3A, the horizontal electrodes constituting the flexible display 130 can measure a capacitance value generated in the horizontal direction of the flexible display 130 according to the hovering gesture. For example, the pitch between the lateral electrodes is about 4 mm, and the width of the lateral electrodes is about 1 mm.

3 (b), the vertical electrodes constituting the flexible display 130 can measure a capacitance value generated in the longitudinal direction of the flexible display 130 according to the hovering gesture. For example, it can be assumed that the distance between the vertical electrodes is about 4 mm, and the width of the vertical electrodes is about 2 mm.

And FIG. 3 (c) is a coordinate value showing the capacitance values measured through the transverse electrodes and the vertical electrodes. In the above coordinates, each number in the x-axis represents transverse electrodes and vertical electrodes in Figs. 3 (a) and 3 (b), respectively. For example, "4" on the x-axis represents "transverse electrode 4 ". The "12" on the x-axis represents the "vertical electrode 12 ". Each of the numbers on the y-axis represents capacitance values that can be measured at the transverse electrodes and the longitudinal electrodes. The unit of the capacitance value may be pF (pico-pel).

3C, the graphs can be divided into four sequences 331, 332, 333, and 334 according to the distance between the input tool 30 and the flexible display 130. The series 331 indicates that the input tool 30 and the flexible display 130 are in contact with each other. The series 332, 333, and 334 indicate that the distances from the input tool 30 to the flexible display 130 are 3 mm, 5 mm, and 10 mm, respectively.

In FIG. 3 (c), the graphs 311, 312, 313 and 314 show capacitance values measured through the transverse electrodes according to the four series. Also, the graphs 321, 322, 323, and 324 represent capacitance values measured through the vertical electrodes according to the four series.

In the graph 311, the capacitance value can be measured and displayed as a peak point at a position 40 where the input tool 30 and the flexible display 130 are in contact with each other. Thus, the electronic device 100 may determine its location 40 as a hovering point, a position at which the user is to select as a hovering gesture.

On the other hand, in the graph 312, as the input tool 30 and the flexible display 130 are spaced apart by a certain distance, capacitance values are measured at peak points at positions 41 and 42 that are different from the position the user wants to select as the hovering gesture . In this case, the electronic device 100 may determine an accurate hovering point corresponding to the user's hovering gesture on the flexible display 130 based on the measured values at the two locations 41,42.

4 is a block diagram of an electronic device 100 in accordance with one embodiment of the present disclosure.

The electronic device 100 of FIG. 4 is represented by a smart phone, but is not limited thereto. For example, the electronic device 100 of FIG. 4 may be a tablet personal computer, a mobile phone, a video phone, an e-book reader, a netbook computer, a workstation, a personal digital assistant (PDA), a portable multimedia player (PMP), an MP3 player, a mobile medical device, a camera, or a wearable device. A wearable device may be of the type of accessory (eg a watch, a ring, a bracelet, a bracelet, a necklace, a spectacles or a head-mounted-device (HMD), a fabric or an integral garment For example, a skin pad or tattoo), or a bio-implantable (e.g., implantable circuit).

In another embodiment, the electronic device 100 may be a home appliance. Home appliances include, for example, televisions, digital video disk players, audio, refrigerators, air conditioners, vacuum cleaners, washing machines, air cleaners, set- top boxes, home automation controls panel, a security control panel, a TV box (eg, Samsung HomeSync ™, Apple TV ™, or Google TV ™), a game console (eg Xbox ™, PlayStation ™), an electronic dictionary, an electronic key, a camcorder, And electronic frames.

In another embodiment, the electronic device 100 may be used in a variety of medical devices (e.g., various portable medical measurement devices such as a blood glucose meter, a heart rate meter, a blood pressure meter, or a body temperature meter), a magnetic resonance angiography (MRA) Navigation systems, global navigation satellite systems (GNSS), event data recorders (EDRs), flight data recorders (FDRs), automotive infotainment systems, (ATMs) (automatic teller's (UPS) systems), airborne electronic devices (eg, infotainment devices), marine electronic devices (eg marine navigation devices, gyro compass, etc.), avionics, security devices, head units for vehicles, industrial or home robots, a point of sale of a store or an internet of things such as a light bulb, various sensors, an electric or gas meter, a sprinkler device, a fire alarm, a thermostat, a streetlight, Of the requester (toaster), exercise equipment, hot water tank, a heater, boiler, etc.) may include at least one.

In yet another embodiment, the electronic device 100 may be a piece of furniture or a part of a building / structure, an electronic board, an electronic signature receiving device, a projector, For example, water, electricity, gas, or radio wave measuring instruments, etc.).

The electronic device 100 may be one or more of the various devices described above and may include new electronic devices as the technology evolves. In particular, the electronic device 100 of the present disclosure may include a hinge in the folding axis 20, a folding structure embodied in a flexible material.

Referring to FIG. 4, the electronic device 100 may include at least one of the status sensing unit 110, the processor 120, and the flexible display 130.

The state sensing unit 110 may sense the folding state of the electronic device 100. [ For example, the status detecting unit 100 may detect whether the electronic device 100 is in an unfolded state or a folded state. In addition, the status sensing unit 110 may sense whether the electronic device 100 is in the folded state or the outfolded state. In addition, the state sensing unit 110 may sense the folding angle as the folding state of the electronic device 100.

The status detecting unit 110 may detect the folding status using various sensors. For example, the state sensing unit 110 may include a plurality of bend sensors arranged on the flexible display 130 or on the Bjel side of the electronic device 100. A bend sensor refers to a sensor that can bend itself and has a resistance value that varies depending on the degree of bending. The bend sensor can be implemented in various forms such as an optical fiber bending sensor, a pressure sensor, a strain gauge, and the like. The state detection unit 110 may detect whether the electronic device is folded by applying a voltage to the bend sensor and sensing a change in the resistance value. If the change in the resistance value is large, it can be judged that the folding is made more. In addition, when the electronic device 100 includes a hinge unit, the state sensing unit 110 may detect a folding state using a contact sensor, a light receiving sensor, a hall sensor, a magnetic sensor, or the like provided in the hinge unit .

When the state sensing unit 110 determines the folding state of the electronic device 100, the state sensing unit 110 may provide the determination result to the processor 120. [ In this case, the processor 120 can determine the folding state of the electronic device 100 according to the output of the state sensing unit 110 without having to separately determine the folding state of the electronic device 100. [ Alternatively, the processor 120 may determine the folding state of the electronic device 100 by providing information about the folding state or the sensing information of the state sensing sensor to the processor 120 in the state sensing unit 110. [

The flexible display 130 may include a display panel that displays a screen on which information is displayed. In addition, the flexible display 130 may include at least one of a touch panel and a pen recognition panel.

The flexible display 130 may be curved or folded asymmetrically or symmetrically.

The flexible display 130 may display a screen displaying information processed by the OS (Operating System) or information to be processed, which is driven in the electronic device 100. For example, the flexible display 130 can display an execution screen of an application processed by the OS, a lock screen, a background screen, an application list screen, and the like.

The flexible display 130 may have a function of detecting not only the touch position, the touch pressure, the touch speed, and the touched area according to the touch gesture, which is the contact touch, but also the duration of the touch. In addition, the flexible display 130 may have a function of detecting a hovering position and a hovering area according to a non-contact touching hovering gesture as well as a duration of hovering.

The processor 120 may control the overall operation of the electronic device 100. For example, the processor 120 may perform operations such as executing and controlling an operating system, processing various data, controlling each component of the electronic device 100, and the like. According to one embodiment, the processor 120 may drive a plurality of different OSs. In general, OS manufacturers can provide various OSs (for example, an OS for a smart phone, an OS for a tablet, an OS for a computer, etc.) according to a screen size of a target device in which an OS is driven. Accordingly, the processor 120 may provide different OSs in the first area and the second area based on the position or folding state of the folding axis of the electronic device 100, and so on.

The processor 120 may determine a hovering point corresponding to the hovering gesture based on the first hovering position and the second hovering position sensed on the flexible display 130 in accordance with the user's hovering gesture on the flexible display 130 .

In one embodiment, the processor 120 corrects at least one position sensed in the folding region 13, in response to the hovering gesture of the user, on the folding region 13 including the folding axis 20 to correspond to the hovering gesture , And control the flexible display 130 to display the UI at the hovering point.

In another embodiment, the processor 120 corrects a plurality of hovering positions sensed in the folding area as the movement of the user's hovering gesture is made on the folding area 13 comprising the folding axis 20 A plurality of hovering points can be determined. Then, the processor 120 may determine a movement term of the hovering gesture based on the determined plurality of hovering points.

In another embodiment, when the processor 120 moves the user's hovering gesture using the input tool on the folded area 13 including the folding axis 20, the processor 120 determines the hovering gesture The direction of movement can be determined.

Although the state sensing unit 110 is illustrated as being separate from the processor 120 and the flexible display 130 in FIG. 4, the state sensing unit 110 may include a function performed by the processor 120 . For example, the processor 120 may receive sensing results directly from sensors in the flexible display 130 to sense the folding condition.

5 is a flow diagram illustrating a method of processing gestures of electronic device 100, in accordance with one embodiment of the present disclosure.

Referring to FIG. 5, in step S501, the electronic device 100 may be folded in by a user or an external force.

In step S502, as the electronic device 100 is folded, the electronic device 100 can sense the folded state of the electronic device 100. [ In this case, when the electronic device 100 is folded with respect to a plurality of axes, the electronic device 100 can sense a plurality of infolding states, respectively. Particularly, due to the external force of the user, when the electronic device 100 is folded in an arbitrary direction, the electronic device 100 can detect a plurality of folding states in a plurality of folding lines along the arbitrary direction have.

The electronic device 100 can correct the hovering position based on the in folding state and determine the folding area that is necessary to determine the hovering point. When a plurality of in folding states are detected, the electronic device 100 can determine a plurality of in folding regions based on a plurality of folding lines.

Herein, the in folding area may mean a folding area in which the electronic device 100 is in the folded state.

In step S504, in the in-folded state, the electronic device 100 may sense the user's hovering gesture on the flexible display 130. [

In step S505, the electronic device 100 may determine whether a hovering gesture has been detected in the folded area.

If the hovering gesture is sensed within the folded area (S505-Y), then in step S506, the electronic device 100 may determine the hovering point by correcting the sensed hovering position in accordance with the hovering gesture. On the other hand, if the hovering gesture is sensed outside the in-folding area (S505-N), then in step S507, the electronic device 100 may determine the hovering position sensed according to the hovering gesture as a hovering point.

FIG. 6 is a diagram illustrating a method for determining a hovering point in an electronic device 100 in an in folded state, in accordance with an embodiment of the present disclosure.

When the in-folding state of the electronic device 100 is sensed by the state sensing unit 110, the processor 120 may determine the in-folding area of the electronic device 100. [ Folding region can be determined to be within a certain distance of both side regions with respect to the folding axis. The size of the folding area can be determined differently depending on the folding state, that is, the folding angle. Next, when a hovering gesture is sensed on the flexible display 130, the processor 120 may determine whether there are a plurality of hovering positions for which the sensed value (e.g., capacitance value) has a peak point according to the hovering gesture. In addition, the processor 120 may determine whether the plurality of hovering positions are within a folding area.

If it is determined that the plurality of hovering positions are within the folding area, the processor 120 may determine a hovering point using the values associated with the plurality of hovering positions.

For example, as shown in FIG. 6 (a), peak point 601 and peak point 602 may be measured according to a hovering gesture. In this case, the processor 120 calculates the difference between the values Ca and Cb detected at the peak points 601 and 602, Pa and Pb, which are distance values from the folding axis to the hovering position at which the peak point is measured, The value k can be used to determine the hovering point Ph1 corresponding to the hovering gesture. This can be expressed as follows.

[Equation 1]

Ph1 = f (Ca, Cb, Pa, Pb, k)

In Equation (1), f () denotes a function implemented to calculate a hovering point with Ca, Cb, Pa, Pb, k as input variables. For example, if the user linearly performs the hovering gesture, the peak point is changed as shown in (b-1) in FIG. 6, (b-2) in FIG. 6, . 6 (b-2), when the capacitance values of the peak points measured in the first area and the second area are equal to each other or substantially equal to each other, the processor 120 sets the position of the folding axis to a hovering point You can decide.

On the other hand, when the hovering gesture moves continuously, the processor 120 may further perform a filtering operation so that the hovering point changes continuously. That is, the processor 120 determines that the hovering point continuously changes along the hovering position where the peak point is changed in (b-1), (b-2) , And further perform a filtering operation so that the gap between the hovering positions becomes linear.

6, the method of determining Ph according to the capacitance values Ca and Cb at the peak points 601 and 602 can be classified as follows.

1) When the Ca value and the Cb value are the same or almost the same

In this case, if the Pa value and the Pb value are the same or substantially the same, the hovering point may be a position where "Ph1 = 0 ". Here, the position of "Ph1 = 0" may be on the folding axis 20.

Further, in this case, when the value of Pa is larger than the value of Pb, the processor 120 may determine the hovering point to be a specific position of the first region where Pa is measured in the folding region. At this time, the hovering point may be a position shifted from the folding axis in the direction of the first region by a ratio of the Pa-Pb value to the Pa value.

Further, in this case, if the value of Pa is much larger than the value of Pb, the processor 120 can determine whether the hovering gesture is a hovering gesture using a plurality of stylus pens. If it is determined that the hovering gesture is a multi-pen hovering, the processor 120 may determine that the first position and the second position are selected, respectively, and perform a specific function according to the hovering gesture. On the other hand, if it is determined that it is not a multiplying hovering gesture, the processor 120 may generate an error event.

2) Cb value is larger than Ca value

In this case, if the Pa value and the Pb value are the same or substantially the same, the processor 120 may determine the hovering point to be the specific position of the first area where Pa is measured within the folding area. In this case, the hovering point may be a position shifted from the folding axis in the direction of the first region by a ratio of the Ca-Cb value. At this time, the Ca and Cb values can be calculated as a log scale value.

In this case, when the Pb value is larger than the Pb value, the processor 120 may determine the hovering point to be a specific position of the first region where the Pa is measured within the folding region. A position shifted from the folding axis in the direction of the first region depending on the ratio of the -Pb value to the Pa value or the ratio of the Ca-Cb value to the Ca value. At this time, the ratio calculated from Pa and Pb and the ratio calculated from Ca and Cb can be applied to the hovering point calculation at approximately the same ratio. At this time, the Ca and Cb values can be calculated as a log scale value.

Also, in this case, if the value of Pa is much greater than the value of Pb, the processor 120 may determine whether the hovering gesture is a multiplying hovering gesture. If it is determined that the hovering gesture is a multipulse hovering, the processor 120 may determine that the first hovering position and the second hovering position are selected, respectively, and process various data corresponding thereto. On the other hand, if it is determined that it is not a multiplying hovering gesture, the processor 120 may generate an error event.

FIG. 7 is a diagram illustrating a method by which an electronic device 100 determines a hovering point, in accordance with an embodiment of the present disclosure.

The graphs of FIG. 7 (a) and the graphs on the coordinates are the same as those of FIG. 3 (c), and a detailed description thereof will be omitted.

When a hovering gesture is sensed on the flexible display 130, the processor 120 may measure the capacitance values through the transverse electrodes and the longitudinal electrodes.

7 (b) is a table showing the capacitance values measured at the transverse electrodes.

The table 701 shows capacitance values measured at the transverse electrodes when the input tool 30 and the flexible display 130 are in contact with each other. In this case, the peak point may be measured on the folding axis 20 at 0.125 pF (701-1). Thus, the hovering point can be located on the folding axis 20.

In the table 702, the capacitance values measured at the transverse electrodes when the distance from the input tool to the flexible display 130 is 3 mm. In this case, the values of the peak points Ca and Cb may be approximately equal to 0.027142 pF (702-1) and 0.02027679 pF (702-2), respectively, and the distances Pa and Pb The values can each be reduced by 8 mm from the folding axis. 6, the Ca value and the Cb value are almost the same, and the Pa value and the Pb value are the same, so that the hovering point can be located on the folding axis 20. [

In the table 703, the capacitance values measured at the transverse electrodes when the distance from the input tool to the flexible display 130 is 5 mm. In this case, the values of the peak points Ca and Cb may be approximately equal to 0.021191 pF (703-1) and 0.021841 pF (703-2), respectively, and the distances Pa and Pb The values can each be reduced by 8 mm from the folding axis. 6, the Ca value and the Cb value are almost the same, and the Pa value and the Pb value are the same, so that the hovering point can be located on the folding axis 20. [

In the table 704, the capacitance values measured at the transverse electrodes when the distance from the input tool to the flexible display 130 is 10 mm. In this case, the values of the peak points Ca and Cb may be approximately equal to 0.013406 pF (704-1) and 0.014022 pF (704-2), respectively, and the distances Pa and Pb The values can each be reduced by 12 mm from the folding axis. 6, the Ca value and the Cb value are almost the same, and the Pa value and the Pb value are the same, so that the hovering point can be located on the folding axis 20. [

FIG. 8 is a diagram illustrating a method by which electronic device 100 determines a hovering point, in accordance with another embodiment of the present disclosure.

8, when a hovering gesture is sensed on the flexible display 130, the processor 120 may determine the hovering point in consideration of the angle between the input tool 30 performing the hovering gesture and the flexible display 130 have.

For example, the processor 120 may determine that the angle [theta] 2 between the input tool 30 and the first area 11 of the flexible display 130 is vertical or nearly vertical (e.g., between 80 and 100) , A hovering point can be determined by weighting the position 801 of the transverse electrode where the peak point Ca is measured in the first region 11. For example, even when the peak point Ca measured in the first region and the peak point Cb measured in the second region are the same, the processor 120 selects the first region 11 as the hovering gesture by the user And determine the position 801 of the transverse electrode whose peak point Ca is measured in the first region 11 as a hovering point. This can be expressed as follows.

&Quot; (2) "

Ph2 = f (Pa, Ca, k)

The processor 120 may utilize the sensed sensed value on the flexible display 130 in accordance with the hovering gesture to determine the angle between the input tool 30 and the flexible display 130. [ For example, according to the hovering gesture, the input tool 30 may be parallel or nearly parallel to the second region 12 of the flexible display 130. In this case, the peak points measured in the second region 12 can be measured more than the peak point measured in the first region 11. Accordingly, the processor 120 can determine the position and tilting angle of the input tool using the peak point measured in the first region 11 and the rate of peak points measured in the second region 12 and the degree of detection at the peak point Can be predicted.

8, a method for determining the hovering point corresponding to the hovering gesture can be distinguished as follows.

1) When the Ph2 hovering point according to [Equation 2] and the Ph1 hovering point according to [Equation 1] are different from each other and all are located in one of the first region 11 and the second region 12

In this case, the processor 120 may determine a final hovering point as an intermediate position between the hovering point derived from the Ph1 equation and the hovering point derived from the Ph2 equation.

2) Ph1 hovering point according to [Equation 2] and Ph1 hovering point according to [Equation 1] are different, one of which is located in the first region 11 and the other is located in the second region 12, If you are located in

In this case, if the angle between the input tool 30 for the hovering gesture and the second region 12 is -10 ° to + 10 °, the processor 120 determines the hovering point derived from the Ph 1 equation as the final hovering point . On the other hand, if the angle between the input tool 30 for the hovering gesture and the second region 12 is out of the range of -10 degrees to +10 degrees, then the processor 120 determines the final hovering point derived from the Ph2 equation You can decide by point.

3) One of the Ph2 hovering point according to [Equation 2] and the Ph1 hovering point according to [Equation 1] is located on the folding axis and the other is located in the first region 11

In this case, if the angle between the input tool for the hovering gesture and the second region 12 is between -10 degrees and +10 degrees, the processor 120 determines the hovering point derived from the Ph1 equation (Equation 1) Can be determined as the final hovering point. On the other hand, if the angle between the input tool 30 for the hovering gesture and the second region 12 is out of the range of -10 degrees to +10 degrees, then the processor 120 determines The hovering point can be determined as the final hovering point.

9A and 9B are diagrams illustrating a method by which electronic device 100 determines a hovering point, in accordance with another embodiment of the present disclosure.

9A, when the folding state is sensed 901, the processor 120 determines the folding area 13 based on the folding state and measures at least one transverse electrode located in the folding area 13 (902). ≪ / RTI > For example, the processor 120 may adjust the amp gain value for a measurement of at least one transverse electrode that is affected by the hovering gesture in the folding area 13 to occur at at least one transverse electrode The inversion of the capacitance value can be prevented. At this time, the magnitude of the at least one transverse electrode and the amplifier gain to be controlled by the amplifier gain value may vary depending on the folding angle of the electronic device 100. [

9B, a graph 901 shows a graph of measured values measured at the transverse electrodes according to the hovering gesture on the folded area 13 before the amp gain value is adjusted. Graph 902 also shows a graph of measured values measured on the transverse electrodes according to the hovering gesture on the folded area 13 after the amp gain value is adjusted.

The hovering positions at which the capacitance values of the peak points 901-1 and 901-2 are measured in the graph 901 may be different from the position at which the user selects the hovering gesture. On the other hand, in the graph 902, the hovering position at which the capacitance value of the peak point 902-1 is measured may be the same position as the position the user wants to select with the hovering gesture.

FIG. 10 is a diagram illustrating a method by which an electronic device 100 determines a hovering point, in accordance with another embodiment of the present disclosure.

In Fig. 10, the transverse electrodes can measure the capacitance value generated according to the hovering gesture. As the transverse electrode is adjacent to the folding axis 20, it is affected by the hovering gesture on the folding region 13. Therefore, when the electronic device 100 is in the folding state, some of the transverse electrodes adjacent to the folding axis 20 are turned off to prevent peak points of the capacitance values of the transverse electrodes at positions not related to the hovering point can do.

For example, as shown in FIG. 10 (a), the processor 120 maintains the off state by skipping a part of the transverse electrodes by one line with respect to the folding axis 20 so that the influence of the transverse electrodes according to the hovering gesture Can be minimized. Accordingly, the transverse electrode 2, the transverse electrode 4, and the transverse electrode 6 are maintained in the ON state, and the transverse electrode 1, the transverse electrode 3, the transverse electrode 5, and the transverse electrode 7 can be maintained in the OFF state.

10 (b) is a view showing a state in which the horizontal electrodes are selectively turned off in a side view of the flexible display 130 in FIG. 10 (a). 10 (b), the transverse electrode 2, the transverse electrode 4, and the transverse electrode 6 are maintained in the ON state, and the transverse electrode 3 and the transverse electrode 5 can be maintained in the OFF state.

11 is a diagram illustrating a method by which electronic device 100 determines a hovering point, in accordance with another embodiment of the present disclosure.

When a hovering gesture is performed by a user, the direction of movement of the hovering gesture may be empirically consistent. Also, the hovering gesture is rarely jumping to a different location, and the speed of movement of the hovering gesture is empirically consistent or tends to be within a certain range. Accordingly, the processor 120 can determine the hovering point according to the new hovering gesture using the characteristics of the hovering gesture described above and the history on which the hovering gesture was performed.

11, the electronic device 100 determines a hovering point 112 determined at time t and a hovering point 111 determined at time t-1 to determine a hovering point 113 at time t + 1 Can be used. For example, the electronic device 100 may determine that the hovering gesture travel direction 1101 or the hovering gesture movement speed of the hovering gesture on the line of travel 1101 of the predetermined hovering points 111, 112 is consistent, The hovering point 113 at the time point t + 1 can be determined in consideration of the moving speed.

12 is a diagram illustrating a method by which electronic device 100 determines a hovering point in accordance with a folding angle, in accordance with another embodiment of the present disclosure.

12, the electronic device 100 can determine the hovering point in consideration of the folding angle [theta] 3 of the electronic device 100. [

For example, as shown in Figure 12 (a), if the folding angle [theta] 3 of the electronic device 100 is less than 45 [deg.], The electronic device 100 may skip the calculation of the hovering point. That is, the electronic device 100 may control so that selection according to the hovering gesture is not performed.

As another example, if the folding angle [theta] 3 of the electronic device 100 is 45 degrees or more and less than 90 degrees, as in Figure 12 (b), the electronic device 100 uses the position determined as the hovering point at the previous point The exact hovering point can be determined. For example, the electronic device 100 may determine a final hovering point as a location that is an average point between a location determined as the hovering point at the current point of time and a location determined as the hovering point at the previous at least one point in time.

As another example, when the folding angle [theta] 3 of the electronic device 100 is 90 [deg.], As shown in Fig. 12C, the electronic device 100 is calculated in accordance with the above- The hovering point can be determined as the final hovering point.

In the same manner, as shown in Fig. 12D, when the folding angle [theta] 3 of the electronic device 100 is more than 90 degrees and less than 180 degrees, A hovering point calculated according to Equation (1) and a hovering point calculated according to Equation (2) to determine a final hovering point.

Alternatively, the electronic device 100 in accordance with the present disclosure may determine the hovering point based on the curvature of the folding axis of the electronic device 100. For example, the electronic device 100 may be configured such that the curvature at which the electronic device 100 is bent is so great that the position of Pa and Pb is located on the curvature, or the curvature at which the electronic device 100 is bent is very small, The hovering point can be calculated by dividing the case where the position of Pb is located outside the curvature.

In addition, the electronic device 100 may store in advance a table for correcting the hovering position according to the hovering gesture. In this case, various kinds of detection waveforms corresponding to the folding angle, the height of the hovering gesture, the position of the hovering gesture, and the tilting angle of the input tool according to the hovering gesture are stored in the table, and the correction value of the hovering position . In this case, the electronic device 100 can use the table to determine an accurate hovering point according to the hovering gesture.

13 is a side view of an electronic device 100 for illustrating an outfolding state, in accordance with one embodiment of the present disclosure.

In Fig. 13 (a), the electronic device 100 can be out-bent by being bent outward. In the outfolding state, the electronic device 100 may sense the user's hovering gesture on the folding area 13 of the flexible display 130. [ In FIG. 13A, reference numeral 1301 denotes a graph of a sensing value (for example, a capacitance value or an electromagnetic field value) measured on the touch panel of the flexible display 130 according to a hovering gesture on the folding area 13. In this case, if the sensing value of the peak point 1301-1 is measured at the transverse electrode located on the folding axis 20, the hovering position where the transverse electrode is located may be determined as a hovering point.

Next, as shown in FIG. 13 (b), the electronic device 100 can detect movement of the hovering gesture on the flexible display 130. In this case, the electronic device 100 may determine that there is a hovering point corresponding to the hovering gesture closer to the folding axis 20 than where the hovering gesture is substantially moved.

In general, as the folding angle [theta] 4 increases, the distance from the input tool for the hovering gesture to the transverse electrodes adjacent to the folding axis 20 increases, and thus the sensing value measured at the adjacent transverse electrodes becomes smaller . Thereby, when the hovering gesture moves on the folding area 13, the hovering point may not immediately follow the position where the hovering gesture is to be selected.

Thus, when the hovering gesture moves on the folding area 13, the electronic device 100 determines that the sensed value of the peak point 1301-2 corrects the measured hovering position so that an accurate hovering point is determined, There is a need to determine the exact hovering point corresponding to the hovering point.

14 is a flow chart illustrating a method for processing a user gesture of the electronic device 100, according to one embodiment of the present disclosure.

Referring to Fig. 14, in step S1401, the electronic device 100 may be folded by a user or an external force.

In step S1402, the electronic device 100 can sense the folded state of the electronic device 100. [ In this case, when the electronic device 100 is folded on the basis of a plurality of axes, the electronic device 100 can sense a plurality of folding states, respectively. In particular, due to the external force of the user, when the electronic device 100 is folded in any direction, the electronic device 100 can respectively detect a plurality of folding states in a plurality of folding lines along the arbitrary direction .

In step S1403, the electronic device 100 can determine whether the electronic device 100 is in the folded state or the outfolded state. In this case, when a plurality of folding states are detected, the electronic device 100 can determine whether each of the plurality of folding states is in a folded state or an outfolded state.

As a result of the determination, if the electronic device 100 is in the folding state (folding in step S1403-), in step S1404, the electronic device 100 corrects the hovering position based on the in folding state, The folding area can be determined.

In step S1405, in the folded state, the electronic device 100 can sense the hovering gesture of the user on the flexible display 130. [

In step S1406, the electronic device 100 may determine whether a hovering gesture has been detected in the folded area.

If the hovering gesture is sensed within the folded area (S1406-Y), then in step S1407, the electronic device 100 may determine the hovering point by correcting the sensed hovering position in accordance with the hovering gesture. On the other hand, if the hovering gesture is sensed outside the in-folding area (S1406-N), then in step S1408, the electronic device 100 may determine the hovering position sensed according to the hovering gesture as a hovering point.

On the other hand, if it is determined in step S1403 that the electronic device 100 is out-folded (S1403-out folding), then in step S1414, the electronic device 100 corrects the hovering position based on the out- It is possible to determine an outfolding area that needs to be determined.

Here, the outfolding region may be a folding region in the outfolding state of the electronic device 100.

In step S1415, in the out-folded state, the electronic device 100 may sense the user's hovering gesture on the flexible display 130. [

In step S1416, the electronic device 100 may determine whether a hovering gesture has been detected in the outfolding area.

If the hovering gesture is sensed within the outfolding area (S1416-Y), then in step S1417, the electronic device 100 may determine the hovering point by correcting the sensed hovering position in accordance with the hovering gesture. On the other hand, if the hovering gesture is sensed outside the in-folding area (S1416-N), then in step S1418, the electronic device 100 may determine the hovering position sensed according to the hovering gesture as a hovering point.

15 is a diagram illustrating a method for determining a hovering point in an electronic device 100 in an outfolded state, in accordance with an embodiment of the present disclosure.

In the state sensing unit 110, when the outfolding state of the electronic device 100 is sensed, the processor 120 can determine an outfolding area of the electronic device 100. [

Next, when a movement of the hovering gesture is detected on the flexible display 130, the processor 120 performs a hovering operation using a hovering position having a sensed value (e.g., a capacitance value) and a peak point measured according to the moving hovering gesture Points can be determined.

For example, as shown in Figure 15 (a), peak point 1501 may be measured as the hovering gesture moves. In this case, the processor 120 calculates the distance value Pa from the folding axis to the head point of the sensing waveform, the distance from the folding axis to the hovering position where the peak point 1501 is measured, The hovering point Ph3 corresponding to the hovering gesture can be determined using the distance value P2 from the axis to the trailing point of the sensed waveform and the variable value k according to the folding angle of the electronic device 100. [ This can be expressed as follows.

&Quot; (3) "

Ph3 = f (Ca, Pa, P1, P2, k)

In Equation (3), the processor 120 can gently deform the sensed waveform measured according to the hovering gesture, as shown in Fig. 15 (b). That is, the processor 120 may modify the sensed waveform by amplifying the sensed values measured at the transverse electrodes adjacent to the folding axis. The processor 120 may then determine a hovering point Ph3 corresponding to the hovering gesture based on the peak point 1502 of the modified sensing waveform.

Thus, as the user moves the hovering gesture, the hovering point corresponding to the hovering gesture is moved gently.

On the other hand, in the electronic device 10 in the out-folded state, when the user linearly performs the hovering gesture, the peak point is changed from (c-1) (c-3). In this case, the processor 120 may further perform a filtering operation so that the hovering point changes continuously. For example, the processor 120 may gently change the sensed waveform measured at (c-1) and (c-3) in FIG. Using the peak points measured in (c-2) in FIG. 15 and the peak points obtained in the slowly changing sensing waveform in FIG. 15 (c-1) and FIG. 6 The filtering operation can be further performed such that the points change continuously and the gap between the hovering positions becomes linear.

16 is a block diagram of an electronic device 100 in accordance with another embodiment of the present disclosure.

16, the electronic device 100 includes a processor 1010, a display portion 1020, a memory 1030, a sensor portion 1035, a communication portion 1040, a video processor 1060, an audio processor 1065, And may include at least one of a user input unit 1050, a microphone unit 1070, an image sensing unit 1075, a speaker unit 1080, and a motion sensing unit 1085.

The user input unit 1050 may sense a first hovering position and a second hovering position on the flexible display 130 according to the hovering gesture of the user. The processor 1010 may determine a hovering point corresponding to the hovering gesture based on the first and second positions sensed on the flexible display 130. Alternatively, the processor 1010 may correct the hovering position sensed according to the user's hovering gesture on the folded area of the flexible display 130, to determine a hovering point corresponding to the hovering gesture on the folding area.

The processor 1010 includes at least one of a RAM (Random Access Memory) 1011, a ROM (Read Only Memory) 1012, a CPU 1013, a GPU (Graphic Processing Unit) 1014, and a bus 1015 can do. The RAM 1011, the ROM 1012, the CPU 1013, and the GPU 1014 may be connected to each other via the bus 1015.

The CPU 1013 accesses the memory 1030 and performs booting using the operating system stored in the memory 1030. [ Then, various operations are performed using various programs, contents, data stored in the memory 1030, and the like.

A ROM 1012 stores a command set for booting the system and the like. For example, when a turn-on command is inputted and power is supplied, the electronic device 100 copies the operating system (OS) stored in the memory 1030 to the RAM 1011 according to a command stored in the ROM 1012 , The system can be booted by running the OS. When the booting is completed, the CPU 1013 copies various programs stored in the memory 1030 to the RAM 1011, executes the program copied to the RAM 1011, and performs various operations. When the booting of the electronic device 100 is completed, the GPU 1014 displays a UI screen in an area of the display unit 1020. [ In addition, the screen generated by the GPU 1014 may be provided to the display unit 1020 and displayed in each area of the display unit 1020, respectively.

The display unit 1020 may include a controller (not shown) for controlling the display panel 1021. The display panel 1021 is implemented with various types of displays such as a liquid crystal display (LCD), an organic light emitting diode (OLED) display, an active matrix organic light-emitting diode (AM-OLED), and a plasma display panel . The display panel 1021 can be embodied as being flexible, transparent or wearable.

The display unit 1020 may be provided with a flexible display 130 that can be combined with at least one of the touch panel 1052 and the pen recognition panel 1053 to detect a contact touch or a non-contact touch.

 For example, the flexible display 130 may include an integrated module in which at least one of the display panel 1021, the touch panel 1052, and the pen recognition panel 1053 is coupled in a laminated structure.

The flexible display 130 may sense a hovering gesture, which is a contact touch and a non-contact touch. At this time, the flexible display 130 can detect not only a user's finger but also a contact touch and a non-contact touch using an input tool or the like.

The memory 1030 may include at least one of an internal memory (not shown) and an external memory (not shown).

The built-in memory may be a volatile memory (for example, a dynamic RAM (DRAM), a static random access memory (SRAM), a synchronous dynamic RAM (SDRAM), or the like), a nonvolatile memory (e.g., an OTPROM ), A PROM (Programmable ROM), an EPROM (Erasable and Programmable ROM), an EEPROM (Electrically Erasable and Programmable ROM), a Mask ROM, a Flash ROM etc.), a hard disk drive (HDD), or a solid state drive .

According to one embodiment, the processor 1010 may load and process instructions or data received from at least one of the non-volatile memory or other components into the volatile memory. In addition, the processor 1010 may store data received or generated from other components in a non-volatile memory.

The external memory may store at least one of CF (Compact Flash), SD (Secure Digital), Micro-SD (Micro Secure Digital), Mini-SD (Mini Secure Digital), xD .

The memory 1030 may store various programs and data used in the operation of the electronic device 100. [ For example, the memory 1030 may temporarily or semi-permanently store at least a part of the content to be displayed on the lock screen. In this disclosure, the memory 1030 may store information or tables necessary to determine the hovering point corresponding to the user's hovering gesture.

The sensor unit 1035 can sense the folding state of the electronic device 100. [ For example, the sensor unit 1035 can detect a folded state or an unfolded state by using a Hall sensor, a magnetic sensor, or the like provided in the folding structure.

The sensor unit 1035 can measure the bending or folding angle (or the spreading angle) of the electronic device 100. Further, the sensor unit 1035 can detect the position of the folding axis, which is a reference for the electronic device 100 to bend or fold. In addition, the sensor unit 1035 can detect a collapsed state by disposing a state detection sensor at a position close to each other due to bending or folding of the electronic device 100. The status detection sensor may include at least one of a proximity sensor, an illuminance sensor, a magnetic sensor, a hall sensor, a touch sensor, a bending sensor, and an infrared sensor, or a combination thereof.

In another embodiment, some of the functionality of the sensor portion 1035 may be performed in the processor 1010. [ For example, the sensor unit 1035 provides the measured various sensed values to the processor 1010, and the processor 1010 detects the position of the folding axis using the sensing value provided from the sensor unit 1035, The user may sense the folded state of the portable terminal 100.

The sensor unit 1035 can detect the folding state of the housing of the electronic device 100 using a Hall sensor or a magnetic sensor provided in the folding structure of the electronic device 100. [ Further, when the electronic device 100 has a hinge structure, the folding angle can be measured in the hinge structure. Alternatively, the sensor unit 1035 can detect a bent state or a folded state by disposing a state detection sensor at a position close to each other due to bending or folding of the housing of the electronic device 100. The status detection sensor may include at least one of a proximity sensor, an illuminance sensor, a magnetic sensor, a hall sensor, a touch sensor, a bending sensor, and an infrared sensor, or a combination thereof. Also, the sensor unit 1035 can detect the position of a folding line, which is a reference for bending or folding of the housing.

The communication unit 1040 can perform communication with various types of external devices according to various types of communication methods. The communication unit 1040 may include at least one of a Wi-Fi chip 1041, a Bluetooth chip 10422, a wireless communication chip 1043, and an NFC chip 1044. [ The processor 1010 can send and receive calls and text messages to and from various external devices using the communication unit 1040. [

The Wi-Fi chip 1041 and the Bluetooth chip 1042 can perform communication using the WiFi method and the Bluetooth method, respectively. In the case of using the Wi-Fi chip 1041 or the Bluetooth chip 1042, various connection information such as an SSID and a session key may be transmitted and received first, and communication information may be used to transmit and receive various information. The wireless communication chip 1043 refers to a chip that performs communication according to various communication standards such as IEEE, ZigBee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution) The NFC chip 1044 refers to a chip operating in an NFC (Near Field Communication) mode using 13.56 MHz band among various RF-ID frequency bands such as 135 kHz, 13.56 MHz, 433 MHz, 860 to 960 MHz, 2.45 GHz,

The video processor 1060 can process the content received through the communication unit 1040 or the video data included in the content stored in the memory 1030. The video processor 1060 may perform various image processing such as decoding, scaling, noise filtering, frame rate conversion, resolution conversion, etc. on the video data.

The audio processor 1065 can process the content received through the communication unit 1040 or the audio data included in the content stored in the memory 1030. [ In the audio processor 1065, various processes such as decoding or amplification of audio data, noise filtering, and the like can be performed.

The processor 1010 may execute the playback program for the multimedia content to play the content by driving the video processor 1060 and the audio processor 1065. [ The speaker unit 1080 can output the audio data generated by the audio processor 1065.

The user input unit 1050 can receive various commands from the user. The user input unit 1050 may include at least one of a key 1051, a touch panel 1052, and a pen recognition panel 1053. [

The touch panel 1052 can detect a user's gesture and output an event value corresponding to the sensed gesture. According to one embodiment, the touch panel 1052 is capable of receiving a user gesture that includes one of the following: a hovering gesture of a user, a tap gesture, a touch and hold gesture, a double tap gesture, a drag gesture, a panning gesture, have. When the touch panel 1052 is combined with the display panel 1021 to form a touch screen (not shown), the touch screen may be implemented by various types of touch sensors such as an electrostatic type, a pressure sensitive type, and a piezoelectric type.

The electrostatic type is a method of calculating the touch coordinates by sensing the minute electricity generated by the user's body when a part of the user's body is touched on the touch screen surface by using a dielectric coated on the surface of the touch screen. The depressurization type includes two electrode plates built in the touch screen. When the user touches the screen, the upper and lower plates of the touched point contact each other to sense current flow, and the touch coordinates are calculated. A touch event occurring on a touch screen can be generated by a finger of a person, but can also be generated by a conductive material capable of applying a capacitance change.

The keys 1051 may include various types of keys, such as mechanical buttons, wheels, etc., formed in various areas such as the front or side of the body exterior of the electronic device 100,

The pen recognition panel 1053 senses the proximity input or touch input of the pen according to the operation of the user's touch pen (e.g., a stylus pen, a digitizer pen) Touch events can be output. The pen recognition panel 1053 can be implemented, for example, to accommodate an active method or a passive method depending on the input method of the touch pen. In the active method, a battery or a coil is incorporated in the touch pen, and the pen recognition panel 1053 can sense the touch or the proximity input according to the proximity of the touch pen or the change in the intensity of the electromagnetic field due to the touch. On the other hand, the passive method can detect the proximity or the touch input of the touch pen by using the conductive property of the touch pen itself without the need for a separate internal element.

One example of an active scheme is an EMR scheme. In this case, the pen recognition panel 1053 includes an electromagnetic induction coil sensor (not shown) having a grid structure and an electronic signal processor (not shown) for sequentially providing an AC signal having a predetermined frequency to each loop coil of the electromagnetic induction coil sensor ). ≪ / RTI > When a pen incorporating a resonance circuit exists in the vicinity of the loop coil of the pen recognition panel 1053, the magnetic field transmitted from the loop coil generates a current based on mutual electromagnetic induction in the resonance circuit in the pen. Based on this current, an induction magnetic field is generated from the coils constituting the resonance circuit in the pen. The pen recognition panel 1053 detects the induction magnetic field in the loop coil in the signal reception state, The touch position can be detected. The pen recognition panel 1053 may be provided at a lower portion of the display panel 1021 with an area capable of covering a display area of the display panel 1021, for example.

The microphone unit 1070 may receive a user voice or other sound and convert the voice data into audio data. The processor 1010 may use the user's voice input through the microphone unit 1070 in a call operation or convert the user's voice into audio data and store the audio data in the memory 1030. [

The image pickup section 1075 can capture a still image or a moving image under the control of the user. The imaging unit 1075 may be implemented as a plurality of cameras such as a front camera and a rear camera.

When the image pickup unit 1075 and the microphone unit 1070 are provided, the processor 1010 performs a control operation in accordance with a user's voice input through the microphone unit 1070 or a user motion recognized by the image pickup unit 1075 It is possible. For example, the electronic device 100 may operate in a motion control mode or a voice control mode. When operating in the motion control mode, the processor 1010 activates the image pickup unit 1075 to pick up a user, track a user's motion change, and perform a corresponding control operation. When operating in the voice control mode, the processor 1010 may operate in a voice recognition mode for analyzing the user voice input through the microphone unit 1070 and performing a control operation according to the analyzed user voice.

The motion detection unit 1085 can detect movement of the main body of the electronic device 100. The electronic device 100 may be rotated or tilted in various directions. At this time, the motion sensing unit 1085 can sense motion characteristics such as a rotation direction, an angle, and a tilt using at least one of various sensors such as a geomagnetic sensor, a gyro sensor, and an acceleration sensor.

16, a USB port through which a USB connector can be connected in the electronic device 100, various external input ports for connecting with various external terminals such as a headset, a mouse, a LAN, etc., a DMB A DMB chip for receiving and processing a digital multimedia broadcasting (DMB) signal, and various sensors.

The names of components of the electronic device 100 described above may be different. Further, the electronic device 100 according to the present disclosure may be configured to include at least one of the above-described components, and some components may be omitted or further include other additional components.

The processor 1010 of FIG. 16 may correspond to the processor 120 of FIG. 4, and the sensor unit 1035 of FIG. 16 may correspond to the state sensing unit 110 of FIG. 16 may be combined with at least one of the touch panel 1052 and the pen recognition panel 1053 of FIG. 16 to correspond to the flexible display 130 of FIG.

17 is a flow diagram illustrating a method for processing a user gesture of the electronic device 100, according to another embodiment of the present disclosure.

In step S1701, the electronic device 100 can sense the first position in the first area and the second position in the second area, according to the hovering gesture of the user on the flexible display.

In step S1702, the electronic device 100 may determine a hovering point corresponding to the hovering gesture based on the sensed first and second positions.

18 is a flow diagram illustrating a method for processing a user gesture of the electronic device 100, according to another embodiment of the present disclosure.

In step S1801, the electronic device 100 can detect the folded state of the electronic device 100. [ The folding state can be, for example, an inbending state in which the electronic device 100 is bent inward, or an outbending state in which the electronic device 100 is bent out.

In step S1802, the electronic device 100 can determine the folding area of the flexible display according to the folding state of the electronic device 100. [ The folding area can increase together as the folding angle of the electronic device 100 increases. For example, the folding area may increase linearly or nonlinearly as the folding angle increases, or may increase stepwise with respect to the thresholds as the folding angle increases.

In step S1803, the electronic device 100 may determine whether the user's hovering gesture is detected.

If the user's hovering gesture is detected (S1803-Y), then in step S1804, the electronic device 100 may determine whether the user's hovering gesture was performed within the folding area or outside the folding area.

If it is determined in step S1805 that the hovering gesture is performed within the folding area (in step S1804-folding area), the electronic device 100 senses the first hovering position in the first area, The second hovering position can be detected. Then, in step S1806, the electronic device 100 may determine a hovering point corresponding to the hovering gesture based on the first hovering position and the second hovering position.

On the other hand, if it is determined that the hovering gesture has been performed outside the folding area (S1804 - outside the folding area), then in step S1815, the electronic device 100 determines the third hovering position in the first area or second area according to the hovering gesture Can be detected. Then, in step S1816, the electronic device 100 may determine the detected third hovering position as a hovering point.

Figs. 19 and 20 are views showing a screen in which the electronic device 100 displays the UI in the folding area, according to an embodiment of the present disclosure.

In one embodiment, in Figures 19 (a) and 19 (b), the electronic device 100 may be in the inbending state bent inward. In such a situation, the processor 120 may sense at least one hovering position in the folding area 13, according to the hovering gesture of the user, on the folding area 13 of the flexible display 130. Next, the processor 120 may determine a hovering point corresponding to the hovering gesture in accordance with the above-described methods, based on the sensed at least one hovering position.

In one example, in a flexible display 130 that is divided into a first region 11 and a second region 12 with respect to a folding axis 20, the processor 120 may generate a first region 11 and a second region 12 according to the hovering gesture of the user. 11, and sense a second position in the second region 12. The first position can be detected in the first region 11, Next, the processor 120 may determine a hovering point corresponding to the hovering gesture based on the sensed first and second positions.

As another example, the processor 120 may sense at least one position in the folded region, depending on the hovering gesture of the user. Next, the processor 120 may correct the sensed position and determine a hovering point corresponding to the hovering gesture.

In this case, the processor 120 may control the flexible display 130 such that a UI (User Interface) is displayed at a hovering point determined in the folding area 13.

For example, as shown in FIG. 19A, the processor 120 may newly display an indicator (e.g., a cursor, a pointer, an insertion marker) 1901 indicating a position designated by the user, To control the flexible display 130 to move and display it. Alternatively, as shown in FIG. 19 (b), the processor 120 can display a new item (e.g., icon, content, etc.) 1902 that can interact with the user, The display 130 can be controlled. In this case, the flexible display 130 may display the item 1902 such that the center area of the item 1902 is located at the hovering point, or may be displayed on one side or one corner of the item 1902 (e.g., May be displayed at the hovering point. When the item 1902 is displayed, the user may input a touch gesture for selecting it. Processor 120 may detect a touch gesture that selects item 1902 on folding area 13. [ If a user's touch gesture is detected, in response, the processor 120 may execute a function corresponding to the selected item 1902. [

In another embodiment, in Figures 20 (a) and 20 (b), the electronic device 100 may be out-bent in outward bending. In this situation, when the user's hovering gesture is input on the folding area 13 of the flexible display 130, the processor 120 may sense at least one hovering position in the folding area 13. [ Next, the processor 120 may determine a hovering point corresponding to the hovering gesture based on the sensed at least one hovering position and control the flexible display 130 to display a UI (User Interface) at the determined hovering point have.

For example, as shown in FIG. 20A, the processor 120 controls the flexible display 130 to display an indicator 2001 indicating the position designated by the user, or the like, . Alternatively, as shown in FIG. 20 (b), the processor 120 may control the flexible display 130 to display a new item 2002 or the like that can be interacted with the user, or to move it from another position. In this case, the flexible display 130 may display an item such that the center area of the item 2002 is located at the hovering point, or the one side or one edge (e.g., the upper left position of the item) You can also display items to be placed at points. When the item 2002 is displayed, the processor 120 may sense a touch gesture of the user selecting the item 2002 on the folding area 13. [ In response to the user's touch gesture, the processor 120 may execute a function corresponding to the selected item 2002. [

FIG. 21 is a diagram illustrating a screen displaying a drawing or handwriting in the folding area 13 of the electronic device 100, according to an embodiment of the present disclosure.

In one embodiment, in Figure 21 (a), the electronic device 100 may be in the inbending state bent inward. In such a situation, the processor 120 may detect a plurality of hovering positions in the folding area 13, according to the movement of the user's hovering gesture performing drawing or writing on the folding area 13 of the flexible display 130 . Next, the processor 120 may determine a plurality of hovering points corresponding to the hovering gesture according to the above-described schemes based on the sensed plurality of hovering positions. The processor 120 may control the flexible display 130 to display the drawing result or handwritten result 2101 in the folded area 13 according to the determined hovering points. If the determined hovering points are discrete, the processor 120 may connect the determined hovering points to control the flexible display 130 to display successive drawing results or handwriting results.

In another embodiment, in Figure 21 (b), the electronic device 100 may be out bent in outward bending. In such a situation, the processor 120 detects a plurality of hovering positions in the folding area 13 and, depending on the hovering gesture of the user performing drawing or handwriting on the folding area 13 of the flexible display 130, A plurality of hovering points corresponding to the hovering gesture may be determined based on the plurality of hovering positions that have been determined. The processor 120 may control the flexible display 130 to display a drawing result or handwritten result 2102 in the folded area 13 according to the determined hovering points.

22 and 23 are views showing screens for selecting items displayed in the folding area 13 of the electronic device 100, according to an embodiment of the present disclosure.

In one embodiment, in Figure 22, the electronic device 100 may be in the inbending state bent inward. In this situation, processor 120 may control the flexible display 130 to display a plurality of items (e.g., icons, content, etc.) 2201, as shown in Figure 22 (a). At least a portion 2202 of the plurality of items 2201 may be located in the folding area 13. If a hovering gesture is input on the folding area 13 in this state, the processor 120 may determine a hovering point corresponding to the hovering gesture in the folding area 13, according to the hovering gesture. The processor 120 may then control the flexible display 130 such that the item 2203 located at the hovering point is highlighted and displayed.

In this state, when the user performs a touch to select the highlighted item 2203, the processor 120 displays the touch gesture of the user selecting the highlighted item 2203, as shown in (b) Can be detected.

In response to the user's touch gesture, the processor 120 can execute the function to display the application execution screen 2204 corresponding to the selected item 2203, as shown in (c) of FIG.

In another embodiment, in Figure 23, the electronic device 100 may be in an out-bent state outwardly. In this situation, as shown in Figure 23 (a), the processor 120 may control the flexible display 130 such that a plurality of items 2301 are displayed. At least a portion 2302 of the plurality of items 2301 may be located in the folding area 13. In this case, the processor 120 may determine a hovering point corresponding to the hovering gesture in the folded area 13, according to the hovering gesture on the folding area 13. [ The processor 120 may then control the flexible display 130 such that the item 2303 located at the hovering point is highlighted and displayed.

Next, in (b) of FIG. 23, the processor 120 may sense a touch gesture of the user selecting the highlighted item 2303.

In response to the user's touch gesture, the processor 120 can execute the function to display the application execution screen 2304 corresponding to the selected item 2303, as shown in (c) of FIG.

24 and 25 are diagrams illustrating a method by which electronic device 100 recognizes movement of a hovering gesture, in accordance with one embodiment of the present disclosure.

In one embodiment, referring to Figure 24 (a), the electronic device 100 may be in the inbending state bent inward. In such a situation, the processor 120 may sense the hovering gesture of the moving user using the input tool 30 on the folded area 13 of the flexible display 130. In this case, the processor 120 may sense a plurality of hovering positions in the folding area 13 as the hovering gesture moves. Next, the processor 120 may determine a plurality of hovering points corresponding to the hovering gesture based on the sensed plurality of hovering positions.

The processor 120 may determine the moving direction v1 of the hovering gesture based on the trajectory of the hovering points determined over time.

In addition, the processor 120 may determine the moving direction v1 of the hovering gesture based on the motion characteristics of the input tool 30. For example, the input tool 30 may be provided with an input tool 30, such as a slope of the input tool 30, an acceleration of the input tool 30, or a direction of movement of the input tool 30, Can be detected. The processor 120 may determine the direction of movement (v1) of the hovering gesture based on at least one of a plurality of hovering points and motion characteristics as a function of the movement of the user's hovering gesture.

Meanwhile, the processor 120 may selectively use the motion characteristics of the input tool 30 to determine the moving direction v1 of the hovering gesture. For example, processor 120 typically determines the direction of movement h1 of the hovering gesture based on the trajectory of a plurality of hovering points, but determines the direction of movement of the first area 11 and second If the measured values measured at the electrodes are the same or similar (e.g., within a certain range) according to the hovering gesture sensed in each of the regions 12, the processor 120 uses the motion characteristics of the input tool 30 to determine the hovering gesture Can be determined.

For example, referring to FIG. 24 (b), the processor 120 calculates the measured values m1, m2 (t) at the first region 11 and the second region 12 according to the hovering gesture at time t + The processor 120 determines that one of the measured values measured according to the hovering gesture is a moving hovering gesture corresponding to the hovering gesture It can be determined as a corresponding hovering point. Next, the processor 120 may determine the moving direction v1 of the hovering gesture based on the hovering point determined at the point t and the hovering point determined at the time t + 1.

In another embodiment, referring to Figure 25 (a), the electronic device 100 may be in the inbending state bent inward. In such a situation, the processor 120 may sense the hovering gesture of the moving user using the input tool 30 on the folded area 13 of the flexible display 130. In this case, the processor 120 may sense a plurality of hovering positions in the folding area 13 as the hovering gesture moves. Next, the processor 120 may determine a plurality of hovering points corresponding to the hovering gesture based on the sensed plurality of hovering positions.

The processor 120 may determine the moving direction v2 of the hovering gesture based on the trajectory of the hovering points determined over time.

In addition, the processor 120 may determine the direction of movement of the hovering gesture based on the motion characteristics of the input tool 30.

For example, referring to FIG. 25 (b), the processor 120 calculates the measured values m3 and m4 at the first region 11 and the second region 12 according to the hovering gesture at time t + ) Are the same or similar, the processor 120 determines, based on the motion characteristics of the input tool 30, a measure m3 of the measured values measured according to the hovering gesture to a shifted hovering gesture corresponding to the hovering gesture It can be determined as a corresponding hovering point. Next, the processor 120 may determine the moving direction v2 of the hovering gesture based on the hovering point determined at the point t and the hovering point determined at the time t + 1.

Figs. 26 and 27 are diagrams illustrating screens on which the electronic device 100 displays the UI according to the movement of the hovering gesture, in accordance with one embodiment of the present disclosure.

In one embodiment, referring to Figures 26 (a) and 26 (b), the electronic device 100 may be in the inbending state bent inward. In such a situation, the processor 120 may sense the hovering gesture of the moving user using the input tool 30 on the folded area of the flexible display 130. [ In this case, the processor 120 may sense a plurality of hovering positions in the folding area 13 as the hovering gesture moves. The processor 120 may determine a plurality of hovering points corresponding to the hovering gesture in accordance with the methods described above based on the sensed plurality of hovering positions.

Next, the processor 120 may determine the direction of movement of the hovering gesture based on at least one of a plurality of hovering points and motion characteristics, in accordance with the manner described above in Figs.

The processor 120 may then display a UI or graphic at locations corresponding to the moving hovering gesture based on the determined direction of movement of the hovering gesture.

As an example, if the hovering gesture moves in a certain direction v3, as in Fig. 26 (a), the processor 120 may determine a hovering point corresponding to the hovering gesture at time t. The processor 120 may wait until time t + 1 and determine the direction of movement of the hovering gesture from time t to time t + 1. For example, the direction of movement of the hovering gesture may be, for example, the vector direction from the hovering point determined at time t to the hovering point determined at time t + 1. Then, the processor 120 may display a UI or graphic representing the movement trajectory of the hovering gesture at time t + 1, based on the moving direction of the hovering gesture. In this case, a time delay of (t + 1) - (t) may occur after the execution of the hovering gesture until the display of the UI or the graphic. However, the UI or the graphic may be displayed at a position where the intention of the user performing the hovering gesture is accurately reflected Can be displayed.

As another example, when the hovering gesture moves in a certain direction v4, as shown in FIG. 26 (b), the processor 120 can determine a hovering point corresponding to the hovering gesture at time t. The processor 120 may then determine the direction of movement of the hovering gesture based on the history of the hovering point. For example, the direction of movement of the hovering gesture may be, for example, the vector direction from the hovering point determined at time t-1 to the hovering point determined at time t. Then, the processor 120 may display a UI indicating a movement trajectory of the hovering gesture at time t, based on the moving direction of the hovering gesture. In this case, the time delay can be minimized from the time the hovering gesture is performed until the UI or graphics are displayed, since the history of the hovering points that have been performed is used without any additional time delay according to the hovering gesture.

In yet another embodiment, the schemes described above in Figures 26 (a) and 26 (b) may be used in combination. For example, when the hovering gesture moves in a certain direction, the processor 120 determines whether or not the history of the moving hovering point is accumulated over a certain time (for example, 0.5 seconds to 2 seconds) ) Scheme, a UI or graphic may be displayed at locations corresponding to a moving hovering gesture. When the history of the hovering point is accumulated for a predetermined time or longer, the processor 120 may display a UI or a graphic at positions corresponding to the moving hovering gesture using the above-described method of FIG. 26 (b).

In another embodiment, when the hovering gesture moves in a certain direction, the processor 120 may determine the direction of movement of the hovering gesture based on the motion characteristics of the input tool 30 performing the hovering gesture. For example, the processor 120 may obtain a measurement associated with the tilt direction of the input tool 30 and may determine a travel direction of the hovering gesture based on the obtained measurement. Then, the processor 120 may display a UI or a graphic representing a movement trajectory according to the movement of the hovering gesture, based on the moving direction of the hovering gesture.

In another embodiment, a hovering gesture can be moved from the first area 11 or the second area 12 to the folding area 13 outside the folding area 13 with respect to the folding axis 20. In this case, the processor 120 may determine whether to move the hovering gesture in the folding area 13 based on the direction of movement of the hovering gesture determined in the first area 11 or the second area 12 outside the folding area 13 A UI or a graphic representing a movement trajectory can be displayed. For example, based on the direction of movement of the hovering gesture determined in the first area 11 outside of the folding area 13, the processor 120 may determine the movement trajectory of the processor 120 hovering gesture in the folding area 13, Or graphics may be positioned on the extension of the movement direction.

In another embodiment, referring to Figures 27 (a) and 27 (b), the electronic device 100 may be out-bent in outward bending. In such a situation, the processor 120 may sense the hovering gesture of the moving user using the input tool 30 on the folded area 13 of the flexible display 130. In this case, the processor 120 may sense a plurality of hovering positions in the folding area 13 as the hovering gesture moves. The processor 120 may determine a plurality of hovering points corresponding to the hovering gesture in accordance with the methods described above based on the sensed plurality of hovering positions.

Next, the processor 120 determines the direction of movement of the hovering gesture based on at least one of the plurality of hovering points and motion characteristics, and determines the position of the hovering gesture based on the UI or Graphics can be displayed.

As an example, if the hovering gesture moves in a certain direction v5, as in Fig. 27 (a), the processor 120 may determine a hovering point corresponding to the hovering gesture at time t. The processor 120 may wait until time t + 1 and determine the direction of movement of the hovering gesture from time t to time t + 1. Then, the processor 120 may display a UI or graphic representing the movement trajectory of the hovering gesture at time t, based on the moving direction of the hovering gesture.

As another example, when the hovering gesture moves in a certain direction v6, as shown in FIG. 27 (b), the processor 120 may determine a hovering point corresponding to the hovering gesture at time t. And, if so, the processor 120 may determine the direction of movement of the hovering gesture based on the history of the hovering point. For example, the direction of movement of the hovering gesture may be, for example, the vector direction from the hovering point determined at time t-1 to the hovering point determined at time t. Then, the processor 120 may display a UI or graphic representing the movement trajectory of the hovering gesture at time t, based on the moving direction of the hovering gesture.

In yet another embodiment, the schemes described above in Figures 27 (a) and 27 (b) may be used in combination. For example, until the hovering point of the moving hovering point accumulates for a predetermined time or more, the processor 120 uses the UI (Fig. 27 (a) Can be displayed. When the history of the hovering point is accumulated for a predetermined time or longer, the processor 120 may display a UI or a graphic at positions corresponding to the moving hovering gesture using the above-described method (b) of FIG.

28 is a flow chart illustrating a method for processing a user gesture of the electronic device 100, according to another embodiment of the present disclosure.

In step S2801, the electronic device 100 can determine whether at least one hovering position in the folding zone 13 is detected, according to the hovering gesture of the user, on the folding area 13 including the folding axis 20 have.

In step S2802, when a hovering position is sensed, the electronic device 100 may determine the hovering point corresponding to the hovering gesture by correcting the sensed hovering position.

In step S2803, the electronic device 100 may display a UI or graphic on the determined hovering point.

29 is a flowchart illustrating a method for processing a user gesture of the electronic device 100, according to another embodiment of the present disclosure.

In step S2901, the electronic device 100 can determine whether a plurality of hovering positions are detected in the folding area 10, according to the hovering gesture of the user, on the folding area 13 including the folding axis 20 .

In step S2902, the electronic device 100 can determine a plurality of hovering points by correcting a plurality of hovering positions.

In step S2903, the electronic device 100 can determine the direction of movement of the hovering gesture based on the determined plurality of hovering points.

In this case, the electronic device 100 may display a UI or graphic representing the movement trajectory of the hovering gesture according to the direction of movement of the hovering gesture.

30 is a flow chart illustrating a method of processing a user gesture of the electronic device 100, according to another embodiment of the present disclosure.

In step S3001, the electronic device 100 can sense the hovering gesture of the user using the input tool on the folding area 13. [

In step S3002, when the motion characteristics of the input tool are sensed, the electronic device 100 may determine the direction of movement of the hovering gesture based on the motion characteristics of the input tool.

Embodiments of the invention and all functional operations described herein may be embodied in digital electronic circuitry or in computer software, firmware, or hardware, including structures invented herein and their equivalents, Can be carried out in combination.

Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer-readable medium can include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes any information delivery media, including computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism.

It is to be understood that the foregoing description of the disclosure is for the purpose of illustration only and that those skilled in the art will readily understand that various changes in form and details may be made therein without departing from the spirit and scope of the disclosure will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive.

The scope of the present disclosure is defined by the appended claims rather than the foregoing detailed description, and all changes or modifications that come within the meaning and range of equivalency of the claims, and equivalents thereof, are to be construed as being included within the scope of the present disclosure do.

110: state detection unit 120: processor
130: Flexible display

Claims (20)

A method of processing a gesture of an electronic device,
When the electronic device is folded so that the flexible display of the electronic device is divided into a first area and a second area with respect to the folding axis, when the hovering gesture is input, the hovering gesture is detected in the first area and the second area, Sensing; And
Determining a hovering point corresponding to the hovering gesture based on a sensing location of each of the first region and the second region. The method according to claim 1,
The hovering gesture comprises:
Wherein the step of performing the gesture processing is performed on a folding area including the folding axis of the flexible display. 3. The method of claim 2,
Further comprising sensing a folding state of the electronic device,
The folding region may be formed,
Wherein the determined gesture is determined according to a sensed folding state of the electronic device. The method of claim 3,
The folded state of the electronic device
Wherein the electronic device is in an inbending state in which the electronic device is bent inwardly or in an out-bending state in which the electronic device is bent outward. 3. The method of claim 2,
The size of the folding region may be,
Wherein the gesture processing method increases as the folding angle of the electronic device increases. The method according to claim 1,
Wherein the sensing position in the first region
Wherein the sensed value measured for the hovering gesture among the electrodes in the first region is a position of an electrode having a peak point. A method of processing a gesture of an electronic device,
Sensing at least one hovering position in the folding area when the hovering gesture is input on the folding area including the folding axis when the electronic device is folded; And
And correcting the sensed hovering position to determine a hovering point corresponding to the hovering gesture in the folded region. A method of processing a gesture of an electronic device,
Sensing at least one hovering position in the folding area when the hovering gesture is input on the folding area including the folding axis when the electronic device is folded; And
Determining a hovering point corresponding to the hovering gesture by correcting the sensed hovering position in the folding region, and displaying the UI at the determined hovering point. A method of processing a gesture of an electronic device,
Sensing a plurality of hovering positions on a folded area including a folding axis in a state in which the electronic device is folded;
Correcting the plurality of hovering positions to determine a plurality of hovering points; And
Determining a direction of movement of the hovering gesture on the folded area based on the determined plurality of hovering points. A method of processing a gesture of an electronic device,
Sensing a hovering gesture using an input tool on a folded area including a folding axis in a state where the electronic device is folded; And
And determining a direction of movement of the hovering gesture based on a motion characteristic of the input tool. In an electronic device,
A flexible display capable of being divided into a first area and a second area based on a folding axis; And
Determining a hovering point corresponding to the hovering gesture based on a first position sensed in the first region and a second sensed position in the second region when the hovering gesture is entered in the folded state of the flexible display, And an electronic device. 12. The method of claim 11,
The hovering gesture comprises:
Wherein the folding is performed on a folding area including the folding axis of the flexible display. 13. The method of claim 12,
Further comprising a state sensing unit for sensing a folding state of the electronic device,
The processor comprising:
And determines the folding area according to the detected folding state. 14. The method of claim 13,
The folded state of the electronic device
Wherein the electronic device is in an inbending state in which the electronic device is bent inward, or an out-bending state in which the electronic device is bent outward. 13. The method of claim 12,
The size of the folding region may be,
And increases as the folding angle of the electronic device increases. 12. The method of claim 11,
Wherein the first position comprises:
Wherein the sensed value measured according to the hovering gesture among the electrodes in the first region is a position of an electrode having a peak point. In an electronic device,
A flexible display which is bent based on a folding axis; And
And a processor for determining a hovering point corresponding to the hovering gesture in the folding area by correcting at least one hovering position sensed in the folding area when a hovering gesture is input on the folding area including the folding axis, Device. In an electronic device,
A flexible display which is bent based on a folding axis; And
Controlling the flexible display so that the UI is displayed at a hovering point corresponding to the hovering gesture determined by correcting at least one hovering position sensed in the folding area when a hovering gesture is input on the folding area including the folding axis, ≪ / RTI > In an electronic device,
A flexible display which is bent based on a folding axis; And
Determining a plurality of hovering points by correcting a plurality of hovering positions sensed in the folding region when a movement of the hovering gesture is made on the folding region including the folding axis, and based on the determined plurality of hovering points, ≪ / RTI > determining a term of movement of the gesture. In an electronic device,
A flexible display which is bent based on a folding axis; And
And a processor for determining a direction of movement of the hovering gesture based on a movement characteristic of the input tool when movement of the hovering gesture using the input tool is performed on the folding area including the folding axis.

Images