KR20230119404A - User terminal and control method thereof - Google Patents

Abstract

A user terminal is disclosed. The user terminal comprises: a flexible display; a housing which supports the flexible display; a sensor; and a processor which, based on data acquired through the sensor, acquires state information including at least one of the active screen size of the flexible display and an angle between one area and another area of the flexible display, based on the acquired state information, identifies whether a reversal event that the housing is changed from a first state to a second state and then returns to the first state has occurred, and when the reversal event is identified, based on the current context information of the user terminal, performs a function corresponding to the reversal event. Therefore, a user can control the user terminal through a reversal gesture on the flexible display in environments where direct touch interaction with the screen is not possible.

Description

User terminal and its control method { User terminal and control method thereof }

The present disclosure relates to a user terminal and a control method thereof, and more particularly, to a user terminal having a flexible display and a control method thereof.

Thanks to the development of electronic technology, various types of electronic devices are being developed and supplied.

In particular, in recent years, the spread of smart phones or tablet PCs that can be carried by users is actively progressing. A smart phone or tablet PC mainly has a touch screen, and a user can control functions of an electronic device through a touch input through the touch screen.

However, there is a problem in that touch input is difficult when the user cannot directly touch the screen, such as when the user's hands are dirty or wearing gloves or when the screen is wet.

The present disclosure has been made due to the above-described necessity, and an object of the present disclosure is to provide a user terminal and a control method for performing a control operation through a specific gesture for a flexible display.

A user terminal according to an embodiment of the present disclosure for achieving the above object includes a flexible display, a housing supporting the flexible display, a sensor, and an active device of the flexible display based on data obtained through the sensor. ) Acquiring state information including at least one of a screen size or an angle between one region and another region of the flexible display, and after the housing is changed from a first state to a second state based on the obtained state information, the and a processor that identifies whether a reversal event to return to a first state has occurred and, if it is identified that the reversal event has occurred, performs a function corresponding to the reversal event based on current context information of the user terminal.

On the other hand, a control method of a user terminal including a flexible display and a housing supporting the flexible display according to an embodiment of the present disclosure, based on data obtained through a sensor, determines an active screen size or obtaining state information including at least one of an angle between one region and another region of the flexible display, and after the housing is changed from a first state to a second state based on the obtained state information, the first state identifying whether a reversal event has occurred, and if it is identified that the reversal event has occurred, performing a function corresponding to the reversal event based on current context information of the user terminal.

According to the various embodiments described above, in an environment in which the user cannot directly touch the screen, such as when the user's hand is dirty or wearing gloves, or when the screen is wet, controlling the user terminal through a reverse gesture for the flexible display You can do it.

1A and 1B are diagrams for explaining an implementation example of a user terminal according to an embodiment.
2 is a block diagram illustrating a configuration of a user terminal according to an exemplary embodiment.
3A to 3C and FIGS. 4A to 4E are diagrams for explaining the definition of an active screen according to an exemplary embodiment.
5 is a flowchart illustrating a method for identifying a reversal event according to an exemplary embodiment.
6A to 6E are diagrams for explaining an example in which a housing state is changed.
7A to 7C are diagrams for explaining a method for determining a reversal event according to an exemplary embodiment.
8A and 8B are diagrams for specifically describing a method for determining a reversal event according to an exemplary embodiment.
9A and 9B are diagrams for describing a reversal event according to an exemplary embodiment.
10A to 10C are diagrams for describing a reversal event along different directions according to an exemplary embodiment.
11A and 11B are views for explaining a reversal event according to different velocities according to an exemplary embodiment.
12A to 12C are diagrams for explaining a method for determining a direction of a reversal event using a sensor according to an exemplary embodiment.
13A to 13G are diagrams for explaining use cases according to an exemplary embodiment.
14 is a block diagram specifically illustrating a configuration of a user terminal according to an exemplary embodiment.

Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

Terms used in this specification will be briefly described, and the present disclosure will be described in detail.

The terms used in the embodiments of the present disclosure have been selected from general terms that are currently widely used as much as possible while considering the functions in the present disclosure, but they may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technologies, and the like. . In addition, in a specific case, there is also a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the disclosure. Therefore, terms used in the present disclosure should be defined based on the meaning of the term and the general content of the present disclosure, not simply the name of the term.

In this specification, expressions such as “has,” “can have,” “includes,” or “can include” indicate the existence of a corresponding feature (eg, numerical value, function, operation, or component such as a part). , which does not preclude the existence of additional features.

The expression at least one of A and/or B should be understood to denote either "A" or "B" or "A and B".

Expressions such as "first," "second," "first," or "second," as used herein, may modify various components regardless of order and/or importance, and may refer to one component It is used only to distinguish it from other components and does not limit the corresponding components.

A component (e.g., a first component) is "(operatively or communicatively) coupled with/to" another component (e.g., a second component); When referred to as "connected to", it should be understood that an element may be directly connected to another element, or may be connected through another element (eg, a third element).

The expression “configured to (or configured to)” as used in this disclosure means, depending on the situation, for example, “suitable for,” “having the capacity to.” ," "designed to," "adapted to," "made to," or "capable of." The term "configured (or set) to" may not necessarily mean only "specifically designed to" hardware.

Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, the terms "comprise" or "consist of" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other It should be understood that the presence or addition of features, numbers, steps, operations, components, parts, or combinations thereof is not precluded.

In the present disclosure, a “module” or “unit” performs at least one function or operation, and may be implemented in hardware or software or a combination of hardware and software. In addition, a plurality of "modules" or a plurality of "units" are integrated into at least one module and implemented by at least one processor (not shown), except for "modules" or "units" that need to be implemented with specific hardware. It can be.

Hereinafter, an embodiment of the present disclosure will be described in more detail with reference to the accompanying drawings.

1A and 1B are diagrams for explaining an implementation example of a user terminal according to an embodiment.

The user terminal 100 may be implemented with various portable terminals such as smart phones, mobile phones, tablets, electronic blackboards, navigation devices, portable media players (PMPs), MP3 players, and game machines. However, it is not limited thereto, and other portable medical devices and wearable devices may also be included in the user terminal 100 . However, in the following description, it is assumed that the user terminal 100 is implemented as a smart phone for convenience of description.

According to an embodiment, the user terminal 100 may be implemented to have a flexible display. A flexible display means a display that can be bent, bent, folded, or rolled like paper. Accordingly, the user terminal 100 can be implemented as a rollable device, a bendable device, a foldable device, and the like. For example, the user terminal 100 may be manufactured on a flexible substrate having flexible characteristics such as a plastic substrate, thin glass, or metal foil.

For example, as shown in FIG. 1A , the user terminal 100 may be implemented as a device of various sizes capable of being folded in various directions.

As another example, as shown in FIG. 1B , the user terminal 100 may be implemented as a device of various sizes capable of rolling in various directions.

Meanwhile, the flexible display provided in the user terminal 100 may be implemented to detect various types of touch inputs. For example, the flexible display can detect various types of touch inputs, such as a touch input by a user's hand, a touch input by an input device such as a stylus pen, and a touch input by a specific electrostatic material. Here, the input device may be implemented as a pen-type input device that may be referred to by various terms such as an electronic pen, a stylus pen, and an S-pen, but is not necessarily implemented in a pen-type. For example, it may be implemented to have a blunt or flat body.

In some cases, touch input to the flexible display may be inconvenient. For example, touch input may be inconvenient when the user's hands are dirty and it is difficult to touch the screen, when the user is wearing gloves, or when the screen is small. can Accordingly, various embodiments of inputting various user commands through non-touch gestures based on unique characteristics of the flexible display will be described below.

2 is a block diagram illustrating a configuration of a user terminal according to an exemplary embodiment.

According to FIG. 2 , the user terminal 100 includes a flexible display 110 , a housing 120 , a sensor 130 and a processor 140 .

The flexible display 110 may be implemented as a display including a self-light emitting element or a display including a non-light emitting element and a backlight. For example, LCD (Liquid Crystal Display), OLED (Organic Light Emitting Diodes) display, LED (Light Emitting Diodes), micro LED (micro LED), Mini LED, PDP (Plasma Display Panel), QD (Quantum dot) display , QLED (Quantum dot light-emitting diodes), etc. can be implemented in various types of displays. The display 110 may also include a driving circuit, a backlight unit, and the like that may be implemented in the form of an a-si TFT, a low temperature poly silicon (LTPS) TFT, or an organic TFT (OTFT).

The housing 120 may be implemented to support the flexible display 110 . According to an example, the housing 120 may include a plurality of housings supporting different regions of the flexible display 110 . Here, the housing functions as a kind of case and may be implemented in various shapes according to a folding or rolling method. According to an example, the housing 120 may include first and second housings provided at both ends of the flexible display 110 to accommodate the rolled display and a third housing to support the non-rolled display. According to another example, the housing 120 may be implemented to include a first housing and a second housing supporting each of a plurality of display areas divided according to the folding of the flexible display 110 . In this case, the first housing and the second housing may be connected by a hinge so as to be foldable.

Other housings 120 may include components designed to cover, contain, or support elements such as bearings, or mechanisms such as pumps or wheels. For example, it may include a bearing housing, a motor housing, a wheel housing, and the like.

The sensor 130 may include various types of sensors such as a touch sensor, a proximity sensor, an acceleration sensor, a geomagnetic sensor, a gyro sensor, a pressure sensor, a position sensor, and the like. For example, the touch sensor may be implemented as a capacitive type or a resistive type. In addition, an infrared sensing method, a surface ultrasonic conduction method, an integral tension measurement method, a piezo effect method, and the like may be used to detect a touch manipulation. An acceleration sensor is a sensor that can measure acceleration and direction of acceleration when motion occurs. The gyro sensor is a sensor that detects the angular velocity by measuring the Coriolis force acting in the direction of the velocity when rotational motion occurs. According to the measurement value of the gyro sensor, it is possible to detect in which direction the bending direction has been rotated, so the bending direction can be detected. A geomagnetic sensor is a sensor that detects an azimuth using a 2-axis or 3-axis fluxgate. In addition, it may include a bend sensor that can be bent by itself and has a characteristic that a resistance value varies according to the degree of bending. The bend sensor may be implemented in various forms, such as an optical fiber bending sensor, a pressure sensor, or a strain gauge.

The processor 140 controls the overall operation of the user terminal 100 . Specifically, the processor 140 may be connected to each component of the user terminal 100 to control the overall operation of the electronic device 100 . For example, the processor 140 may be electrically connected to the flexible display 110, the plurality of housings 120, and the sensor 130 to control the overall operation of the user terminal 100. Processor 140 may be composed of one or a plurality of processors.

The processor 140 may perform the operation of the user terminal 100 according to various embodiments by executing at least one instruction stored in a memory (not shown).

According to an embodiment, the processor 140 may include a digital signal processor (DSP), a microprocessor, a graphics processing unit (GPU), an artificial intelligence (AI) processor, and a neural processing unit (NPU) for processing digital image signals. Processing Unit), time controller (TCON), but is not limited to, central processing unit (CPU), micro controller unit (MCU), micro processing unit (MPU), controller (controller), an application processor (AP), a communication processor (CP), or one or more of an ARM processor, or may be defined by the term. In addition, the processor 140 may be implemented in the form of a system on chip (SoC) with a built-in processing algorithm, large scale integration (LSI), application specific integrated circuit (ASIC), or field programmable gate array (FPGA).

According to an embodiment, the processor 140 determines the active screen size of the flexible display 110 or the angle between one area and another area of the flexible display 110 based on the data acquired through the sensor 130. State information including at least one of may be obtained. Here, the active screen means a screen on which an image is currently displayed.

3A to 3C and FIGS. 4A to 4E are diagrams for explaining the definition of an active screen according to an exemplary embodiment.

3A shows an example of a screen 311 and application screens 312 and 313 .

3b and 3c illustrate one and other examples 330, 314, and 315 in which one device provides multiple screens 312 and 313.

4A shows examples 411 and 412 of the active screen, and FIG. 4B shows another example 413 of the active screen.

4C shows an example 414 of an inactive screen.

4D shows an example in which the size of the active screens 415 and 416 is changed according to the rolling of the device.

4E shows an example in which the size of the active screens 417, 418, and 419 is changed according to the folding of the device.

Returning to FIG. 2 , the processor 140 may obtain state change information of a plurality of housings based on the acquired state information.

The processor 140 may identify whether a reversal event in which the relative positions of the plurality of housings are changed from the first state to the second state and then return to the first state occurs. In this case, the processor 140 identifies that the relative positions of the plurality of housings have changed from the first state to the second state when the relative positions of the housings change by more than the first threshold value from the first state, and if the relative positions of the housings change from the second state to the second threshold value or more, the first state It can be identified as returning to . Here, the change of a first threshold value or more may mean a change of a specific threshold range or less while being changed by a first threshold value or more. That is, it may mean that an excessive change above the first threshold value is not additionally generated. In addition, the change of the second threshold value or more may mean that the change is less than a specific threshold range while being changed by the second threshold value or more. That is, it may mean that an excessive change above the second threshold value is not additionally generated. Here, the sizes of the first threshold and the second threshold may be the same, but are not limited thereto and may be different. According to an example, the processor 140 may identify whether an excessive change is additionally generated at or above the first threshold value or the second threshold value based on the speed at which the state changes, etc. A detailed description of this will be described later with reference to the drawings. let it do

Alternatively, the processor 140 identifies that the relative positions of the housings are changed to the second state when the relative positions of the housings are changed to satisfy the first threshold value in the first state, and change to satisfy the second threshold value in the second state, It can be identified as returning to the first state. Here, the change to satisfy the first threshold value means a change within the error range based on the first threshold value, and the change to satisfy the second threshold value means the change within the error range based on the second threshold value. can mean

Here, the above-described first threshold value and second threshold value may be preset values at the time of manufacturing the user terminal 100 or values set/changed by the user. Also, the first threshold value and the second threshold value may be updated based on a user's manipulation history. That is, since the intensity of force or the degree of manipulation may be different for each user, the first threshold value and the second threshold value may be updated to user-customized values based on the user's manipulation history.

Then, when it is determined that a reversal event has occurred, the processor 140 may perform a task or function corresponding to the reversal event based on the current context information of the user terminal 100 . Here, the context information includes various information related to the context of the user terminal 100, such as information on an application being executed, information on a function or task being executed, information on an executable function or task, remaining power information, time information, lighting information, and the like. information may be included. For example, if it is determined that a reversal event has occurred, the processor 140 identifies an application and execution function being executed in the user terminal 100, and performs a function corresponding to the reversal event based on the identified application and execution function. can

Also, the processor 140 may perform a task or function corresponding to a reversal event based on a reversal attribute corresponding to the reversal event. Here, at least one of a reversal direction, a reversal amount, a reversal acceleration, a reversal speed, a reversal frequency, or a reversal position may be included. Here, the inversion frequency may mean the number of times the same state is repeated within a unit time.

According to an embodiment, a first housing in which one area of the flexible display 110 is rolled and accommodated in the plurality of housings 120 and a second housing in which the other area of the flexible display 120 is rolled and accommodated Let us assume the case that includes . In this case, the sensor 130 may include a first motion sensor provided in the first housing and a second motion sensor provided in the second housing. Here, the first motion sensor and the second motion sensor may include various types of sensors capable of sensing motion, such as an acceleration sensor, a geomagnetic sensor, and a gyro sensor.

In this case, the processor 140 changes the active screen size of the flexible display 110 from the first size to the second size as the flexible display 110 rolls, and then returns to the threshold size based on the first size. , it may be determined that a reversal event has occurred. Here, the reversal event may be an event in which the distance between the first housing and the second housing increases and then decreases, or decreases and then increases.

Also, the processor 140 determines at least one of a movement amount and a movement direction of the first housing and the second housing as the flexible display 110 is rolled based on sensing data obtained through the first motion sensor and the second motion sensor. It is possible to identify movement information that includes. Also, the processor 140 may perform a function corresponding to a reversal event based on the identified movement information and current context information of the user terminal 100 .

According to another embodiment, it is assumed that the plurality of housings 120 include a first housing implemented to support one area of the flexible display and a second housing implemented to support the remaining area of the flexible display 110. . In this case, the sensor 130 may include a first motion sensor provided in the first housing and a second motion sensor provided in the second housing.

In this case, as the flexible display 110 is folded, the processor 140 changes the angle between one area and the other area from the first angle to the second angle and returns to the threshold angle based on the first angle. It may be determined that a reversal event has occurred based on at least one of an event or an event in which the exposure screen size of the flexible display 110 is changed from the first size to the second size and then returns to the threshold size based on the first size. there is. Here, the reversal event may be an event in which at least one of the angle or the distance between the first housing and the second housing increases and then decreases or decreases and then increases.

Also, the processor 140 determines at least one of a movement amount and a movement direction of the first housing and the second housing as the flexible display 110 is folded based on sensing data obtained through the first motion sensor and the second motion sensor. It is possible to identify movement information that includes. Also, the processor 140 may perform a function corresponding to a reversal event based on the identified movement information and current context information of the user terminal 100 .

5 is a flowchart illustrating a method for identifying a reversal event according to an exemplary embodiment.

According to FIG. 5 , when the housing state is changed (S505), the processor 140 identifies the current time (S510). In this case, the identified time can be stored as a parameter corresponding to the start time. In this case, time may be stored in units of milliseconds, but is not limited thereto.

Next, the processor 140 identifies a new housing state according to the action of changing the housing state (S515). In this case, when it is identified that the housing state is changed to a preset state (S520:Y), the processor 140 identifies whether the housing state is reversed (S525). In this case, in step S520, it may be identified whether the housing state is changed to a preset state based on the first threshold value.

When the housing state is identified as being reversed (S525: Y), the processor 140 identifies a new housing state according to an action of changing the housing state after the housing state is reversed (S530).

When it is identified that the housing state has changed to a preset state (S530:Y), the processor 140 identifies whether the difference between the current time and the time identified in step S510 is less than a threshold time (S540). In this case, in step S530, it may be identified whether the housing state is changed to a preset state based on the second threshold value.

If the difference between the current time and the time identified in step S510 is less than the threshold time Δt, the processor 140 identifies that a reversal event has occurred and controls the user terminal 100 to perform a corresponding function (S545). Here, the threshold time may be the maximum time for identifying a reversal event.

6A to 6E are diagrams for explaining an example in which a housing state is changed.

6A is a diagram for explaining a change in a housing state when the user terminal 100 is rolled in a non-rolled state. As shown in FIG. 6A , states of the housings 611 , 612 , and 613 may be different when the user terminal 100 is not rolled and when the user terminal 100 is rolled.

6B is a diagram for explaining a change in a housing state when the user terminal 100 is folded from an unfolded state. As shown in FIG. 6B, the states of the housings 621 and 622 may be different when the user terminal 100 is not folded and when the user terminal 100 is folded. can be different

As shown in FIGS. 6C and 6D , when the size of the active screens 641 and 641 is changed according to the user's rolling manipulation or the size of the active screens 651 and 652 is changed according to the folding manipulation, FIGS. 6A and 6B The state of the housing described above is also changed.

In addition, as shown in FIGS. 6E and 6F, when angles α1 and α2 between one area and another area of the flexible display are changed according to a user's folding operation, the state of the housing described in FIGS. 6A and 6B may also change. can

7A to 7C are diagrams for explaining a method for determining a reversal event according to an exemplary embodiment.

According to FIG. 7A , the processor 140 may first select a monitoring factor for monitoring the housing state (S711). For example, you can check the setting mode for selecting a monitoring factor.

When the housing state is monitored based on the size of the active screen (S712), the processor 140 acquires the first screen size (S1), which is the current screen size (S713), and the second screen size, which is the screen size after a predetermined time interval. 2 The screen size (S2) is obtained (S714). Next, the processor 140 compares the first screen size (S1) and the second screen size (S2), and when the second screen size (S2) is not equal to the first screen size (S1) (S715), specifically The operation may proceed with the operation shown in FIG. 7B.

Meanwhile, when monitoring the housing state based on the folding angle (S716), the processor 140 obtains the first folding angle α1, which is the current folding angle (S717), and obtains the folding angle α1 after a predetermined time interval (S717). 2 The folding angle α2 is obtained (S718). Then, the processor 140 compares the first folding angle α1 and the second folding angle α2, and when the second folding angle α2 is not equal to the first folding angle α1 (S719), specifically The operation may proceed with the operation shown in FIG. 7C.

According to FIG. 7B, the processor 140 identifies the current time as a time stamp (S721). In this case, the identified time can be stored as a parameter corresponding to the start time. In this case, time may be stored in units of milliseconds, but is not limited thereto.

When the size of the active screen is changed (S722), the processor 140 obtains the changed screen size (S723) and checks whether the changed screen size satisfies a first threshold (S724).

When the changed screen size satisfies the first threshold value (S724: Y), the processor 140 identifies whether the screen size is reversed (S725).

When the screen size is identified as being inverted (S725: Y), the processor 140 obtains the screen size (S726) and checks whether the obtained screen size satisfies the second threshold (S727).

When the screen size satisfies the second threshold, the processor 140 reverses the current time and the time difference identified in step S721 if the time difference identified in step S721 is less than the threshold time Δt (S728:Y). It is identified that an event has occurred (S729).

Meanwhile, according to FIG. 7C, the processor 140 identifies the current time as a time stamp (S731). In this case, the identified time can be stored as a parameter corresponding to the start time. In this case, time may be stored in units of milliseconds, but is not limited thereto.

When the folding angle is changed (S732), the processor 140 obtains the changed folding angle (S733) and checks whether the changed folding angle satisfies a first threshold value (S734).

When the changed folding angle satisfies the first threshold value (S734:Y), the processor 140 identifies whether the folding angle is reversed (S735).

When the folding angle is identified as being inverted (S735:Y), the processor 140 obtains the folding angle (S736) and determines whether the obtained folding angle satisfies the second threshold (S737).

When the folding angle satisfies the second threshold, the processor 140 reverses the current time and the time difference identified in step S731 if the time difference identified in step S731 is less than the threshold time Δt (S738:Y). It is identified that an event has occurred (S739).

8A and 8B are diagrams for specifically describing a method for determining a reversal event according to an exemplary embodiment.

According to FIG. 8A, the processor 140 identifies the current time as a time stamp (S811).

When the housing state is changed (S812), the processor 140 obtains the first housing state S1 (S813) and obtains the second housing state S2 after the first critical time Δt1 (S814). Here, the housing state may be the size of the active screen.

The processor 140 identifies whether the second housing state S2 obtained after the first threshold time Δt1 is the same as the first housing state S1 previously obtained (S815).

If the second housing state S2 is not the same as the first housing state S1 (S815:N), the processor 140 determines that the second housing state S2 is the first housing state S1. It is identified whether it is greater than (S816).

If the second housing state S2 is greater than the previously acquired first housing state S1 (S816:Y), the processor 140 identifies the housing state as having an increasing tendency (S817), The third housing state IS1 is obtained (S818), and the fourth housing state IS2 increased after the second threshold time Δt2 is obtained (S819).

Subsequently, the processor 140 identifies whether the fourth housing state IS2 is the same as the third housing state IS1 (S820). If the fourth housing state IS2 is not equal to the third housing state IS1 (S820:N), it is identified whether the fourth housing state IS2 is smaller than the third housing state IS1 (S820:N). S821). That is, the processor 140 determines the size of the active screen after a user operation for increasing the size of the active screen is identified, and then the increased size decreases again (ie, the fourth housing state IS2 is smaller than the third housing state IS1). case), it is possible to determine whether the user is performing a reversal operation.

If the fourth housing state (IS2) is less than the third housing state (IS1) (S821:Y), it is determined whether the difference between the current time and the time identified in S811 is less than the fourth threshold time (Δt4) (S822 ). If the difference between the current time and the time identified in S811 is smaller than the fourth threshold time Δt4 (S822:Y), it is identified that the housing state is reversed (S823).

Meanwhile, if the second housing state S2 is smaller than the previously obtained first housing state S1 (S816:N), the processor 140 identifies the housing state as having a decreasing tendency (S824), The reduced fifth housing state DS1 is obtained (S825), and the reduced sixth housing state DS2 after the third threshold time Δt3 is obtained (S826).

Subsequently, the processor 140 identifies whether the sixth housing state DS2 is the same as the fifth housing state DS1 (S827). If the sixth housing state DS2 is not equal to the fifth housing state DS1 (S827:N), it is determined whether the sixth housing state DS2 is greater than the fifth housing state DS1 (S828). ). That is, the processor 140 identifies a user operation for reducing the size of the active screen, and then increases the reduced size again (ie, the sixth housing state DS2 is larger than the fifth housing state DS1). case), it is possible to determine whether the user is performing a reversal operation.

If the sixth housing state DS2 is greater than the fifth housing state DS1 (S828:Y), it is determined whether the difference between the current time and the time identified in S811 is less than the fourth threshold time Δt4 (S822). . When the difference between the current time and the time identified in S811 is smaller than the fourth threshold time Δt4 (S822:Y), it is identified that the housing state is reversed (S823).

According to FIG. 8B, the processor 140 identifies the current time as a time stamp (S831).

When the housing state is changed (S832), the processor 140 obtains the first folding angle α1 (S833) and obtains the second folding angle α2 after the first threshold time Δt1 (S834).

The processor 140 identifies whether the second folding angle α2 obtained after the first threshold time Δt1 is equal to the previously obtained first folding angle α1 (S835).

If the second folding angle α2 is not equal to the first folding angle α1 (S835:N), the processor 140 determines that the second folding angle α2 is equal to the first folding angle αS1. It is identified whether it is greater than (S836).

If the second folding angle α2 is greater than the previously obtained first folding angle α1 (S836:Y), the processor 140 identifies the folding angle as having an increasing tendency (S817), and the increased third folding angle α2. The folding angle αA1 is obtained (S838), and the fourth folding angle αA2 increased after the second threshold time Δt2 is obtained (S839).

Subsequently, the processor 140 identifies whether the fourth folding angle αA2 is equal to the third folding angle αA1 (S840). If the fourth folding angle αA2 is not equal to the third folding angle αA1 (S840:N), it is identified whether the fourth folding angle αA2 is smaller than the third folding angle αA1 (S841). . That is, the processor 140 determines that the increased folding angle decreases again after the user manipulation to increase the folding angle is identified (ie, the fourth folding angle αA2 is smaller than the third folding angle αA1). must be identified so that it can be determined whether the user is performing a reversal operation.

If the fourth folding angle αA2 is smaller than the third folding angle αA1 (S841:Y), it is identified whether the difference between the current time and the time identified in S831 is smaller than the fourth threshold time Δt4 (S832) . When the difference between the current time and the time identified in S831 is smaller than the fourth threshold time Δt4 (S842:Y), it is identified that the housing state is reversed (S843).

Meanwhile, if the second folding angle α2 is smaller than the previously obtained first folding angle α1 (S836:N), the processor 140 identifies that the folding angle has a decreasing tendency (S844), and decreases. The fifth folding angle αD1 is obtained (S845), and the reduced sixth folding angle αD2 after the third threshold time (Δt3) is obtained (S846).

Next, the processor 140 identifies whether the sixth folding angle αD2 is equal to the fifth folding angle αD1 (S847). If the sixth folding angle αD2 is not equal to the fifth folding angle αD1 (S847:N), it is determined whether the sixth folding angle αD2 is greater than the fifth folding angle αD1 (S848). That is, the processor 140, after identifying a user manipulation to decrease the folding angle, when the reduced folding angle increases again (ie, when the sixth folding angle αD2 is greater than the fifth folding angle αD1) must be identified so that it can be determined whether the user is performing a reversal operation.

If the sixth folding angle αD2 is greater than the fifth folding angle αD1 (S848:Y), it is determined whether the difference between the current time and the time identified in S841 is less than the fourth threshold time Δt4 (S842). When the difference between the current time and the time identified in S841 is smaller than the fourth threshold time (Δt4) (S842:Y), it is identified that the housing state is reversed (S843).

9A and 9B are diagrams for describing a reversal event according to an exemplary embodiment.

According to FIGS. 9A and 9B , the processor 140 may determine that an inversion event has occurred when the housing state of the user terminal 100 is changed from the first state to the second state and then returns to the first state.

For example, as shown in FIG. 9A , when the user terminal 100 is implemented as a rollable device, the processor 140 changes the size of the active screen from a first state to a second one to satisfy a preset condition according to a user manipulation. When it decreases to the first state and then increases to the first state, it may be determined that a reversal event has occurred. Here, the preset condition may be a case where the second state in which the screen size is reduced and the first state in which the screen size is increased satisfy the first threshold value and the second threshold value, respectively.

As another example, as shown in FIG. 9B , when the user terminal 100 is implemented as a foldable device, the processor 140 changes the folding angle from the first state to the second state according to a user manipulation to satisfy a preset condition. When it is increased and then decreased again to the first state, it may be determined that an inversion event has occurred. Here, the predetermined condition may be a case where the second state in which the folding angle is increased and the first state in which the folding angle is decreased satisfy the first threshold value and the second threshold value, respectively.

10A, 10B, and 10C are views for explaining a reversal event along different directions according to an exemplary embodiment.

According to FIG. 10A , when the housing state is changed (S1005), the processor 140 identifies a movement direction according to the housing state change (S1010). Here, the moving direction may include a rolling direction and a folding direction of the screen based on the plurality of housings.

When the moving direction is identified as the first direction (S1015), the processor 140 may identify whether a reversal event occurs (S1020).

When it is identified that the inversion event has occurred (S1020:Y), the processor 140 may control the user terminal 100 according to the first control mode (S1030).

Meanwhile, when the moving direction is identified as the second direction (S1035), the processor 140 may identify whether a reversal event occurs (S1040).

When it is identified that the inversion event has occurred (S1040:Y), the processor 140 may control the user terminal 100 according to the second control mode (S1045). Here, the second control mode may be a mode that performs different control or different functions from the first control mode.

For example, as shown in FIG. 10B , a case in which one side 1011 of the user terminal 100 is folded in the direction of another side 1012 is a first direction movement, and the other side 1012 is one side 1011. ) direction may be the second direction movement.

As another example, as shown in FIG. 10C , the case where the display screen 1022 of the user terminal 100 is rolled to the second housing 1023 is the movement in the first direction, and the case where the display screen 1022 is rolled to the first housing 1021 is the movement in the first direction. It may be movement in the second direction.

11A and 11B are views for explaining a reversal event according to different velocities according to an exemplary embodiment.

According to an embodiment, the processor 140 may perform different control operations according to the speed of the inversion event.

According to FIG. 11A, when a reversal event occurs (S1105), the processor 140 identifies whether the rate of the reversal event is greater than the level N (S1110), and if the rate of the reversal event is greater than the level N (S1110: Y), an operation corresponding to control mode N may be performed (S1115).

If the speed of the reversal event is less than level N (S1110: N), it identifies whether or not it is greater than the next level N-1, and if the speed of the reversal event is greater than level N-1, the operation corresponding to the control mode N-1 can be performed.

By sequentially performing these operations, it is identified whether or not the minimum level is greater than 1 (S1120), and when the speed of the reversal event is greater than level 1 (S1120: Y), an operation corresponding to control mode 1 can be performed. . Meanwhile, when the speed of the reversal event is less than level 1 (S1120: N), an operation corresponding to control mode 0 may be performed (S1130).

Here, the control modes N, N-1, ...1, and 0 may be modes for performing different control or different functions, respectively.

In this case, the processor 140 may calculate the speed of the inversion event by dividing the change value range by the inversion event occurrence time (eg, Δt described above).

For example, the processor 140 may control the number of page turns according to the speed of the inversion event. For example, as shown in FIG. 11B , when a speed reversal event corresponding to level 1 occurs while P1 and P2 are displayed on the screen, P3 and P4 are displayed on the screen and a speed reversal event corresponding to level 2 occurs. occurs, P11 and P12 can be displayed on the screen.

12A to 12C are diagrams for explaining a method for determining a direction of a reversal event using a sensor according to an exemplary embodiment.

In FIGS. 12A to 12C , a case in which the user terminal 100 is implemented as a foldable device will be described.

As shown in FIG. 12A , motion sensors 1211 and 1221 may be provided in the first housing 1210 and the second housing 1220 of the user terminal 100 , respectively. Here, the motion sensor may include various types of sensors capable of detecting motion, such as a speed sensor, a geomagnetic sensor, and a gyro sensor, as described above.

According to FIG. 12B , the processor 140 may identify a case in which the left first housing 1210 is moved when a change is detected in a sensing value of the first motion sensor 1211 provided in the first housing 1210. .

According to FIG. 12C , the processor 140 may identify a case in which the left first housing 1220 is moved when a change is detected in a sensing value of the second motion sensor 1221 provided in the second housing 1220. .

In this case, of course, various movement states as well as a movement direction can be identified based on the size, variation, and the like of the sensing values acquired through the first motion sensor 1211 and the second motion sensor 1221. For example, a specific OS (operating system) included in the processor 140 may provide an application programming interface (API) to acquire motion parameters and determine a movement direction and/or which part is moved.

According to an example, an acceleration sensor is a sensor capable of measuring acceleration and direction of acceleration when motion occurs. Specifically, the acceleration sensor outputs a sensing value corresponding to the gravitational acceleration that changes according to the inclination of the part to which the sensor is attached. Accordingly, when acceleration sensors are disposed in each of the first and second housings, the direction of movement may be determined by calculating a pitch angle and a roll angle using output values sensed by each acceleration sensor.

As described above, the bending direction may be detected using a gyro sensor or a geomagnetic sensor in addition to the acceleration sensor. The gyro sensor is a sensor that detects the angular velocity by measuring the Coriolis force acting in the direction of the velocity when rotational motion occurs. According to the measured value of the gyro sensor, it is possible to detect in which direction the folding direction has been rotated. A geomagnetic sensor is a sensor that detects an azimuth using a 2-axis or 3-axis fluxgate. The direction of movement may be determined by calculating a yaw angle using a value output from the geomagnetic sensor.

13A to 13G are diagrams for explaining use cases according to an exemplary embodiment.

13A to 13C illustrate an example in which an inversion gesture is used, and it is assumed that the user terminal 100 is implemented as a foldable device.

According to FIGS. 13A to 13C , when a call is received, the user can automatically change the call direction using a reversal gesture. For example, if the user needs to touch the "call button" to receive a call, but the user's hands are dirty or wearing gloves, the screen is wet (in the rain or swimming, etc.), etc. There may be cases where you cannot directly touch . In this case, the call direction can be changed so that the other party can receive a call by using a simple reversal gesture.

FIG. 13D illustrates an example in which a reverse gesture is used, and assumes that the user terminal 100 is implemented as a rollable device.

Referring to FIG. 13D , when a call is received, the user can automatically change the call direction using a reversal gesture. That is, the direction of a call can be changed so that the other party can receive a call by using a simple reversal gesture as shown.

FIG. 13E illustrates an example in which an inversion gesture is used, and assumes that the user terminal 100 is implemented as a foldable device.

Referring to FIG. 13E , a user may change a photographing mode using a reverse gesture while photographing using a camera application. For example, if the user needs to touch the "mode change button" to change the shooting mode, but the user cannot directly touch the screen, such as when the user's hands are dirty or wearing gloves, or the screen is wet. there may be In this case, the shooting mode may be switched from the omnidirectional shooting mode to the selfie mode by using a simple reverse gesture.

13F and 13G show an example in which an inversion gesture is used, and it is assumed that the user terminal 100 is implemented as a foldable device.

According to FIGS. 13F and 13G , a user may move a page using a reverse gesture in reading mode. For example, if the user needs to input a touch such as flick or drag to switch pages, but the user cannot directly touch the screen, such as when the user's hands are dirty or wearing gloves, or the screen is wet. there may be In this case, it is possible to move to the next page or the previous page by using a simple reverse gesture. According to FIG. 13G , after moving to web page 2 by clicking a link in web page 1, it is possible to move to web page 1 again by using a reversal gesture.

14 is a block diagram specifically illustrating a configuration of a user terminal according to an exemplary embodiment.

According to FIG. 14, a user terminal 100' includes a flexible display 110, a housing 120, a sensor 130, a processor 140, a memory 150, a communication interface 160, and a user interface 170. can include Among the components shown in FIG. 12, detailed descriptions of components overlapping with those shown in FIG. 4 will be omitted.

The memory 150 may store data necessary for various embodiments. The memory 150 may be implemented in the form of a memory embedded in the user terminal 100' or in the form of a removable memory in the user terminal 100', depending on the data storage purpose. For example, data for driving the user terminal 100' is stored in a memory embedded in the user terminal 100', and data for extended functions of the user terminal 100' is stored in the user terminal 100'. ) can be stored in a removable memory. On the other hand, in the case of memory embedded in the user terminal 100', volatile memory (eg, DRAM (dynamic RAM), SRAM (static RAM), SDRAM (synchronous dynamic RAM), etc.), non-volatile memory (non-volatile memory) (eg OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (eg NAND flash or NOR flash etc.), a hard drive, or a solid state drive (SSD) In addition, in the case of a memory that is detachable from the user terminal 100', a memory card (eg, CF ( compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.) It may be implemented in the form of a memory (eg, USB memory) or the like.

Of course, the communication interface 160 may be implemented as various interfaces according to the implementation example of the user terminal 100'. For example, the communication interface 160 includes Bluetooth, AP-based Wi-Fi (Wi-Fi, Wireless LAN network), Zigbee, wired/wireless LAN (Local Area Network), WAN (Wide Area Network), Ethernet, IEEE 1394, HDMI (High-Definition Multimedia Interface), USB (Universal Serial Bus), MHL (Mobile High-Definition Link), AES/EBU (Audio Engineering Society/ European Broadcasting Union), Optical Communication may be performed with an external device, an external storage medium (eg, a USB memory), an external server (eg, a web hard drive), etc. through a communication method such as , coaxial, or the like.

The user interface 170 may be implemented as a device such as a button, touch pad, mouse, or keyboard, or may be implemented as a touch screen capable of simultaneously performing the above-described display function and manipulation input function.

A speaker (not shown) may be configured to output not only various kinds of audio data, but also various notification sounds or voice messages. The processor 140 may control a speaker to output information corresponding to a UI screen or various notifications in the form of audio according to various embodiments of the present disclosure.

A microphone (not shown) is a component for receiving a user's voice or other sounds and converting them into audio data. However, according to another embodiment, the user terminal 100 ′ may receive a user voice input through an external device through the communication interface 150 .

A camera (not shown) may be turned on according to a predetermined event to perform photographing. A camera (not shown) may convert a captured image into an electrical signal and generate image data based on the converted signal. For example, an object may be converted into an electrical image signal through a charge coupled device (CCD), and the image signal thus converted may be amplified and converted into a digital signal and then signal processed.

According to various embodiments described above, a user terminal can be controlled through a reversal gesture on a flexible display in an environment where a touch is difficult. For example, various functions such as changing a call direction, turning a page in a reading mode or surfing mode, and switching a camera mode may be performed through a reverse gesture on a flexible display.

Meanwhile, the methods according to various embodiments of the present disclosure described above may be implemented in the form of an application that can be installed in an existing electronic device. Alternatively, the above-described methods according to various embodiments of the present disclosure, for example, identification of a reversal event may be performed using a deep learning-based artificial neural network (or deep artificial neural network), that is, a learning network model.

In addition, the methods according to various embodiments of the present disclosure described above may be implemented only by upgrading software or hardware of an existing electronic device.

In addition, various embodiments of the present disclosure described above may be performed through an embedded server included in the electronic device or an external server of the electronic device.

Meanwhile, according to an exemplary embodiment of the present disclosure, the various embodiments described above may be implemented as software including instructions stored in a machine-readable storage media (eg, a computer). can A device is a device capable of calling a stored command from a storage medium and operating according to the called command, and may include an electronic device (eg, the electronic device A) according to the disclosed embodiments. When a command is executed by a processor, the processor may perform a function corresponding to the command directly or by using other components under the control of the processor. An instruction may include code generated or executed by a compiler or interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' only means that the storage medium does not contain a signal and is tangible, but does not distinguish whether data is stored semi-permanently or temporarily in the storage medium.

Also, according to an embodiment of the present disclosure, the method according to the various embodiments described above may be included in a computer program product and provided. Computer program products may be traded between sellers and buyers as commodities. The computer program product may be distributed in the form of a device-readable storage medium (eg compact disc read only memory (CD-ROM)) or online through an application store (eg Play Store™). In the case of online distribution, at least part of the computer program product may be temporarily stored or temporarily created in a storage medium such as a manufacturer's server, an application store server, or a relay server's memory.

In addition, each of the components (eg, modules or programs) according to various embodiments described above may be composed of a single object or a plurality of entities, and some sub-components among the aforementioned sub-components may be omitted, or other sub-components may be used. Components may be further included in various embodiments. Alternatively or additionally, some components (eg, modules or programs) may be integrated into one entity and perform the same or similar functions performed by each corresponding component prior to integration. According to various embodiments, operations performed by modules, programs, or other components may be executed sequentially, in parallel, repetitively, or heuristically, or at least some operations may be executed in a different order, may be omitted, or other operations may be added. can

Although the preferred embodiments of the present disclosure have been shown and described above, the present disclosure is not limited to the specific embodiments described above, and is common in the technical field belonging to the present disclosure without departing from the gist of the present disclosure claimed in the claims. Of course, various modifications and implementations are possible by those with knowledge of, and these modifications should not be individually understood from the technical spirit or perspective of the present disclosure.

100: user terminal 110: flexible display
120: housing 130: sensor

Claims (20)

In the user terminal,
flexible display;
a housing supporting the flexible display;
sensor; and
obtaining state information including at least one of an active screen size of the flexible display or an angle between one area and another area of the flexible display based on data obtained through the sensor;
Identifying whether or not an inversion event in which the housing returns to the first state after changing from the first state to the second state has occurred based on the obtained state information;
When it is identified that the reversal event has occurred, a processor that performs a function corresponding to the reversal event based on current context information of the user terminal. According to claim 1,
the housing,
It includes a first housing and a second housing,
the processor,
When the relative positions of the first housing and the second housing are changed from the first state to a first threshold value or more, it is identified as being changed to the second state;
When the second state is changed by more than a second threshold value, it is identified that the user terminal has returned to the first state. According to claim 1,
the housing,
a first housing implemented such that one area of the flexible display is rolled and accommodated; and
A second housing implemented such that the remaining area of the flexible display is rolled and accommodated;
the processor,
When the active screen size of the flexible display is changed from a first size to a second size as the flexible display rolls and returns to within a threshold size based on the first size, identifying that the inversion event has occurred ,
The reversal event,
An event in which the distance between the first housing and the second housing increases and then decreases or decreases and then increases. According to claim 3,
The sensor,
a first motion sensor provided in the first housing; and
And a second motion sensor provided in each of the second housings,
the processor,
Movement information including at least one of a movement amount and a movement direction of the first housing and the second housing as the flexible display is rolled based on sensing data obtained through the first motion sensor and the second motion sensor A user terminal that identifies and performs a function corresponding to the reversal event based on the identified movement information and current context information of the user terminal. According to claim 1,
the housing,
a first housing configured to support one area of the flexible display; and
A second housing implemented to support the remaining area of the flexible display; includes,
the processor,
As the flexible display is folded, an event in which an angle between the one area and the remaining area is changed from a first angle to a second angle and then returned to within a critical angle based on the first angle or the flexible display Identifies that the reversal event has occurred based on at least one of events returning within a threshold size based on the first size after the exposure screen size is changed from the first size to the second size;
The reversal event,
An event in which at least one of the angle or the distance between the first housing and the second housing increases and then decreases or decreases and then increases. According to claim 5,
The sensor,
a first motion sensor provided in the first housing; and
And a second motion sensor provided in each of the second housings,
the processor,
Movement information including at least one of a movement amount and a movement direction of the first housing and the second housing as the flexible display is folded based on sensing data obtained through the first motion sensor and the second motion sensor A user terminal that identifies and performs a function corresponding to the reversal event based on the identified movement information and current context information of the user terminal. According to claim 1,
the processor,
and identifying that the reversal event has occurred when a time from when the housing changes to the second state to returning to the first state is within a threshold time. According to claim 1,
the processor,
Performing a task or function corresponding to the reversal event based on a reversal attribute corresponding to the reversal event;
The inversion property is
A user terminal including at least one of a reverse direction, a reverse amount, a reverse acceleration, a reverse speed, a reverse frequency, or a reverse position. According to claim 1,
the processor,
If it is identified that the reversal event has occurred, identifying an application and an execution function running in the user terminal;
A user terminal that performs a function corresponding to the reversal event based on the identified application and execution function. According to claim 1,
the processor,
A user terminal that performs at least one function of changing a call direction, moving a page, or switching a camera mode based on the inversion event. A control method of a user terminal including a flexible display and a housing supporting the flexible display,
obtaining state information including at least one of an active screen size of the flexible display or an angle between one area and another area of the flexible display based on data acquired through a sensor;
identifying whether a reversal event in which the housing returns to the first state after changing from the first state to the second state has occurred based on the acquired state information; and
and performing a function corresponding to the reversal event based on current context information of the user terminal when it is identified that the reversal event has occurred. According to claim 11,
the housing,
It includes a first housing and a second housing,
The step of identifying whether the reversal event has occurred,
When the relative positions of the first housing and the second housing are changed from the first state to a first threshold value or more, it is identified as being changed to the second state;
If the second state is changed by more than a second threshold value, it is identified as returning to the first state. According to claim 11,
the housing,
a first housing implemented such that one area of the flexible display is rolled and accommodated; and
A second housing implemented such that the remaining area of the flexible display is rolled and accommodated;
The step of identifying whether the reversal event has occurred,
When the active screen size of the flexible display changes from a first size to a second size as the flexible display rolls and returns to within a threshold size based on the first size, identifying that the inversion event has occurred ,
The reversal event,
An event in which the distance between the first housing and the second housing increases and then decreases or decreases and then increases. According to claim 13,
The sensor,
a first motion sensor provided in the first housing; and
And a second motion sensor provided in each of the second housings,
The step of performing a function corresponding to the reversal event,
Movement information including at least one of a movement amount and a movement direction of the first housing and the second housing as the flexible display is rolled based on sensing data obtained through the first motion sensor and the second motion sensor and performing a function corresponding to the reversal event based on the identified movement information and current context information of the user terminal. According to claim 11,
the housing,
a first housing configured to support one area of the flexible display; and
A second housing implemented to support the remaining area of the flexible display; includes,
The step of identifying whether the reversal event has occurred,
As the flexible display is folded, an event in which an angle between the one area and the remaining area is changed from a first angle to a second angle and then returned to within a critical angle based on the first angle or the flexible display Identifies that the reversal event has occurred based on at least one of events returning within a threshold size based on the first size after the exposure screen size is changed from the first size to the second size;
The reversal event,
An event in which at least one of an angle or a distance between the first housing and the second housing increases and then decreases or decreases and then increases. According to claim 15,
The sensor,
a first motion sensor provided in the first housing; and
And a second motion sensor provided in each of the second housings,
The step of performing a function corresponding to the reversal event,
Movement information including at least one of a movement amount and a movement direction of the first housing and the second housing as the flexible display is folded based on sensing data acquired through the first motion sensor and the second motion sensor and performing a function corresponding to the reversal event based on the identified movement information and current context information of the user terminal. According to claim 11,
The step of identifying whether the reversal event has occurred,
and identifying that the reversal event has occurred when a time from when the housing is changed to the second state to returning to the first state is within a threshold time. According to claim 11,
The step of performing a function corresponding to the reversal event,
Performing a task or function corresponding to the reversal event based on a reversal attribute corresponding to the reversal event;
The inversion property is
A control method comprising at least one of a reversal direction, a reversal amount, a reversal acceleration, a reversal speed, a reversal frequency, or a reversal position. According to claim 1,
The step of performing a function corresponding to the reversal event,
If it is identified that the reversal event has occurred, identifying an application and an execution function running in the user terminal;
A control method for performing a function corresponding to the reversal event based on the identified application and execution function. According to claim 11,
The step of performing a function corresponding to the reversal event,
A control method for performing at least one function of changing a call direction, moving a page, or switching a camera mode based on the inversion event.

Images